I have an array of keys which lead to post objects for my social network like so /posts/id/(post info)

When I load the posts I load /posts/0 and then /posts/1 etc using the observeSingleEventOfType(.Value) method.

I use a lazyTableView to load 30 at a time and it is quite slow. Is there any way I can use one of the query methods or another way of making it faster even if I have to restructure the data in my JSON tree.

I am coming from Parse re-implementing my app and so far the experience as been quite good. Just this one thing I am a bit stuck on. Thanks in advance for the help!

EDIT:

func loadNext(i: Int)
    {
        // check if exhists
        let ideaPostsRef = Firebase(url: "https://APPURL")
        ideaPostsRef.childByAppendingPath(i.description).observeSingleEventOfType(.Value, withBlock: {
            (snapshot) in
            if i % 29 == 0 && i != 0 && !self.hitNull { return }
            // false if nil
            // true if not nil
            if !(snapshot.value is NSNull)
            {
                let postJSON  = snapshot.value as! [String: AnyObject]
                print("GOT VALID \(postJSON)")
                let post = IdeaPost(message: postJSON["message"] as! String, byUser: postJSON["user"] as! String, withId: i.description)
                post.upvotes = postJSON["upvotes"] as! Int
                self.ideaPostDataSource.append(post)
                self.loadNext(i + 1)
            }
            else
            {
                // doesn't exhist
                print("GOT NULL RETURNING AT \(i)")
                self.doneLoading = true
                self.hitNull = true
                return
            }
        })

This recursive function essentially runs getting the value for key number i from firebase. If it is NSNULL it knows that is the last possible post to load and never does again. If NSNULL doesn't get hit but i % 29 == 0 then it returns as a base case so only 30 posts are loaded at a time (0 indexed). When I set doneLoading to true tableView.reloadData() is called using a property observer.

Here is a sample of what the array I am fetching looks like

"ideaPosts" : [ {
    "id" : 0,
    "message" : "Test",
    "upvotes" : 1,
    "user" : "Anonymous"
  }, {
    "id" : 1,
    "message" : "Test2",
    "upvotes" : 1,
    "user" : "Anonymous"
  } ]
1 upvote
  flag
It'll be a lot easier to help if you show us your code instead of describing it. Include the minimum JSON (as text, not a screenshot) and code to reproduce the problem in your question and we can see how it can be improved. Read more about an MCVE. – Frank van Puffelen
upvote
  flag
Edited to include code explanation – Big_Mac_24

1 Answers 11

up vote 74 down vote accepted

Update: we now also cover this question in an AskFirebase episode.

Loading many items from Firebase doesn't have to be slow, since you can pipeline the requests. But your code is making this impossible, which indeed will lead to suboptimal performance.

In your code, you request an item from the server, wait for that item to return and then load the next one. In a simplified sequence diagram that looks like:

Your app                     Firebase 
                             Database

        -- request item 1 -->
                               S  L
                               e  o
                               r  a
                               v  d
                               e  i
        <-  return item  1 --  r  n
                                  g
        -- request item 2 -->
                               S  L
                               e  o
                               r  a
                               v  d
                               e  i
                               r  n
        <-  return item  2 --     g
        -- request item 3 -->
                 .
                 .
                 .
        -- request item 30-->
                               S  L
                               e  o
                               r  a
                               v  d
                               e  i
                               r  n
                                  g
        <-  return item 30 --

In this scenario you're waiting for 30 times your roundtrip time + 30 times the time it takes to load the data from disk. If (for the sake of simplicity) we say that roundtrips take 1 second and loading an item from disk also takes one second that least to 30 * (1 + 1) = 60 seconds.

In Firebase applications you'll get much better performance if you send all the requests (or at least a reasonable number of them) in one go:

Your app                     Firebase 
                             Database

        -- request item 1 -->
        -- request item 2 -->  S  L
        -- request item 3 -->  e  o
                 .             r  a
                 .             v  d
                 .             e  i
        -- request item 30-->  r  n
                                  g
        <-  return item  1 --     
        <-  return item  2 --      
        <-  return item  3 --
                 .
                 .
                 .
        <-  return item 30 --

If we again assume a 1 second roundtrip and 1 second of loading, you're waiting for 30*1 + 1 = 31 seconds.

So: all requests go through the same connection. Given that, the only difference between get(1), get(2), get(3) and getAll([1,2,3]) is some overhead for the frames.

I set up a jsbin to demonstrate the behavior. The data model is very simple, but it shows off the difference.

function loadVideosSequential(videoIds) {
  if (videoIds.length > 0) {
    db.child('videos').child(videoIds[0]).once('value', snapshot => {
      if (videoIds.length > 1) {
        loadVideosSequential(videoIds.splice(1), callback)
      }
    });
  }
}

function loadVideosParallel(videoIds) {
  Promise.all(
    videoIds.map(id => db.child('videos').child(id).once('value'))
  );
}

For comparison: sequentially loading 64 items takes 3.8 seconds on my system, while loading them pipelined (as the Firebase client does natively) it takes 600ms. The exact numbers will depend on your connection (latency and bandwidth), but the pipelined version should always be significantly faster.

10 upvote
  flag
Nice, Puf! Also, chaining promises (jQuery.whenAll(), q.all(), or Promise.all()) can be very handy here if you need all items loaded, but still want to grab them in parallel, before taking some action. – Kato
4 upvote
  flag
Cool. Didn't even think of that, even though I have been using it. :-) – Frank van Puffelen
upvote
  flag
Am i missing something? You are calling to loadVideosSequential only after the previous "once" fetched the snapshot, similar to the how @Big_Mac_24 does but with different API call. Can you explain the difference please? – Shirane85
1 upvote
  flag
@FrankvanPuffelen You are right from the performance point of view but what if one of these calls didn't return due to any type of error? How can you 'cancel' rest of the pending requests if anyone out of these is failed. In case of sequential requests, we can get to know in code that which request failed. Please share your thoughts. Thanks. – Perry
1 upvote
  flag
"The Promise.all() method [...] rejects with the reason of the first promise that rejects." – pejalo
upvote
  flag
Perry, you wanna use Observables if you wanna cancel the rest. – DarkNeuron
1 upvote
  flag
How we can do Promise.all in android? How we can load all the data in android – Muhammad chhota
upvote
  flag
Please describe this case for Android – Konstantin Konopko

Not the answer you're looking for? Browse other questions tagged or ask your own question.