Scroll Append - Inifinite/Endless Scroll

By Hawkee on Apr 16, 2012

This will automatically append the next page of results as you continue to scroll down a page.

Features:

  • Define a number of pages to append before pausing with a "Show More" button.
  • Float the footer after a single page of results. Reverts back to relative after another page loads.
  • Uses localStorage so the user won't lose their place when clicking away and returning.

Usage:

$(window).scrollAppend({
    url: 'newsfeed.php',
    params: { type: "image", who: "friends" },
    appendTo: "#newsfeed",
    footerClass: "#footer"
});

Here are the CSS classes it uses (.footer_fixed is required for the fixed footer feature):

.scroll_append_more { width: 100%; text-align: center; }
.scroll_append_loading { width: 100%; }
.scroll_append_loading img { display: block; margin-left: auto; margin-right: auto; }
.footer_fixed { position: fixed; bottom: 0px; }

When you've reached the last page of results simply have your server-side script return the word "false" in plain text.

Github repo:
https://github.com/Hawkers/scrollAppend

Demo:
http://hawkee.com/profile/2/

/********************************************************************************
/*
 * scrollAppend (jQuery auto append results)
 * 2012 by Hawkee.com (hawkee@gmail.com)
 *
 * Version 1.5
 * 
 * Requires jQuery 1.7 and jQuery UI 1.8
 *
 * Dual licensed under MIT or GPLv2 licenses
 *   http://en.wikipedia.org/wiki/MIT_License
 *   http://en.wikipedia.org/wiki/GNU_General_Public_License
 *
 *
 *  Options:
 *
 *  url: Where to query for the next page of results.
 *
 *  params: An array of parameters to be passed to the URL
 *
 *  appendTo: The div that will get the next page of results.
 *
 *  callback: Anything that needs to be called once the next page has been appended.
 *
 *  pixelBuffer: Pixes from the bottom of the window before starting to load next page.
 *
 *  pageVar: What will be passed to your server to represent the page number.
 *
 *  expireCacheMins: Number of minutes before cached appends get cleared.
 *
 *  pagesToPause: The number of pages to show before we pause and require a click to continue.
 *
 *  loadingImage: A spinner or image to indicate the next page is loading.
 *
 *  moreText: The text that will be indicate a pause.
 *
 *  disableCache: Always start from the first page, don't cache appends.
 *
 *  footerClass: Set this if you want your footer to appear fixed after the first page of results.
 *
 *  contentClass: Set this if you want your main content bottom-margin to adjust to the floating fixed footer.
 *
 *  footerSpeed: The speed at which your footer slides up after the first page of results.
 *
 * Usage:
 *
 * $(window).scrollAppend({
 *      url: 'newsfeed.php',
 *      params: { type: "image", who: "friends" },
 *      appendTo: "#newsfeed",
 *      footerClass: "#footer"
 *  });
*/

(function($){$.widget("ui.scrollAppend", {

    options:{
        pixelBuffer: 400,
        pageVar: 'p',
        expireCacheMins: 20,
        pagesToPause: 5,
        loadingImage: '/images/loading.gif',
        moreText: 'Show More',
        disableCache: false,
        footerClass: undefined,
        contentClass: undefined,
        footerSpeed: 400
    },

    _create:function() {

        var self = this;
        self.loading = false;
        self.page = 0;
        self.stop = false;
        self.pause = false;
        self.stopFixed = false;
        self.footerUp = false;
        self.lastScrollPos = undefined;
        self.position = undefined;
        self.appearedOnce = false;

        self.options.contentClass = undefined;

        // Get the params and save them as a key so we can store the cache according to the page
        // it was generated on.
        var params_key = self.options.params;
        params_key[self.options.pageVar] = 0;
        var param_key = jQuery.param(params_key);

        self.param_key = param_key;

        // Clears expired localStorage data.
        this.clearOld();

        // Look for cached appends.
        if(!self.options.disableCache) {
            var cache = localStorage.getItem('scroll_'+self.param_key);
            if(cache != undefined) {
                var timestamp = localStorage.getItem('time_'+self.param_key);
                self.page = localStorage.getItem('p_'+self.param_key);
                //console.log("resuming from page: "+self.page+" at "+timestamp);
                $(this.options.appendTo).append(cache);
                if(self.options.callback) self.options.callback.apply();
            }
        }

        // Fix the footer and hide it below the bottom of the page to bring it up after a single screen of results.
        if(self.options.footerClass != undefined) {
            self.setFixed();
        }   

        // See if we're already at the end of the results on the first page.
        self.checkAppend();

        // Determines if we scrolled to the bottom of the page and appends if we still have results.
        $(window).scroll(function () {
            self.checkAppend();
            if(self.options.footerClass != undefined && !self.loading) self.checkDirection();
        });

        $(document).on('click', '#scroll_append_more', function() {
            self.append();
            $(this).remove();
            self.pause = false;
            return false;
        });
    },

    // Checks if we need to append.
    // Checks if we need to slide the footer into view.
    checkAppend: function () {
        var self = this;
        var height = $(document).height() - $(window).height();

        // Check to see if we're within pixelBuffer of the bottom of the window.
        if($(window).scrollTop() >= height - self.options.pixelBuffer) {
            if(!self.loading) {
                if(self.stop) return;
                if(!self.pause) self.append();
            }
        }
    },

    // Slides the footer up or down
    // 'persistent' will only show the footer and not remove it. 
    toggleFooter: function(persistent) {
        var self = this;
        if(self.options.footerClass == undefined) return;
        var contentHeight = $(self.options.footerClass).outerHeight();

        if(!self.footerUp) {
            // Start the footer below the bottom and slide it up into view.
            $(self.options.footerClass).stop(true, true).animate(
                {bottom: 0},
                {duration: self.options.footerSpeed,queue: false}
            );

            // Adjust the page margin to include the footer.
            if(self.options.contentClass != undefined) {
                $(self.options.contentClass).stop(true, true).animate(
                    {'margin-bottom':contentHeight},
                    {duration: self.options.footerSpeed,queue: false}
                );
            }

            self.appearedOnce = true;
            self.footerUp = true;
        }
        else if(!persistent) {

            // Remove the fixed class so the footer stays at the bottom.
            $(self.options.footerClass).stop(true, true).animate(
                {bottom:-contentHeight},
                {duration: self.options.footerSpeed,queue: false}
            );

            // Adjust the page margin to exclude the footer.
            if(self.options.contentClass != undefined) {
                $(self.options.contentClass).stop(true, true).animate(
                    {'margin-bottom': 0},
                    {duration:self.options.footerSpeed,queue:false}
                );
            }   

            self.footerUp = false;
            self.stopFixed = true;
        }
    },

    // Makes the footer fixed and adjusts the margin of the page.
    setFixed: function() {
        var self = this;

        // We don't need to make it fixed if we're paused or stopped.
        if(self.stop || self.pause) return;

        // If we're already fixed then just return.
        if(self.position == 'fixed') return;
        var contentHeight = $(self.options.footerClass).outerHeight();

        $(self.options.footerClass).css('bottom', -contentHeight);

        $(self.options.footerClass).addClass('footer_fixed')
        if(self.options.contentClass != undefined) {
            // Every time the footer is initialized to fixed it'll be under the bottom of the page.
            $(self.options.contentClass).css('margin-bottom', 0);
        }
        self.footerUp = false;
        self.position = 'fixed';
    },

    // Makes the footer relative and adjusts the margin of the page.
    setRelative: function() {
        var self = this;
        if(self.position == 'relative') return;

        $(self.options.footerClass).removeClass('footer_fixed');
        $(self.options.footerClass).css('bottom', 0);
        if(self.options.contentClass != undefined) {
            $(self.options.contentClass).css('margin-bottom', 0);
        }

        // If it has appeared already and the user scrolls up then stop it from showing up except at the more button.
        if(self.appearedOnce) self.stopFixed = true;
        self.position = 'relative';
    },

    // Determines what direction the user is scrolling.
    // If the footer is fixed and the user starts scrolling up set it back to relative so it doesn't
    // eat up the viewport.  If they're scrolling down, set it back to fixed.
    checkDirection: function() {
        var self = this;
        var scrollPos = $(window).scrollTop();
        if(scrollPos < self.lastScrollPos) {
            self.atBottom = false;
            self.setRelative();
        }
        // Set it back to fixed so it doesn't show up for a split second to be pushed down with each append.
        else self.setFixed();
        self.lastScrollPos = scrollPos;
    },

    // Appends the next page
    append: function() {

        var self = this;
        self.loading = true;
        self.page++;

        self.options.params[self.options.pageVar] = self.page;

        var loadingImage;
        if(self.options.loadingImage) {
            $(self.options.appendTo).append("<div id='scroll_append_loading' class='scroll_append_loading'><img src='"+self.options.loadingImage+"'></div>");
        }

        // Only fix the footer a single time after the first page is loaded.
        if(!self.stopFixed) self.toggleFooter();

        $.ajax({
            url: self.options.url,
            data: self.options.params,
            cache: false,
            success: function(html){
                $('#scroll_append_loading').remove();
                if(html == 'false') {
                    self.stop = true;
                }   
                else {
                    $(self.options.appendTo).append(html);

                    // Update the cache for returning to the page.

                    if(!self.options.disableCache) {
                        var old = localStorage.getItem('scroll_'+self.param_key);
                        if(old === null) old = "";
                        self.saveData('scroll_'+self.param_key, old + html);
                        self.saveData('p_'+self.param_key, self.page);
                        var timestamp = Number(new Date());
                        self.saveData('time_'+self.param_key, timestamp);
                    }
                }

                // Check if we need to pause.

                var mod = self.page % self.options.pagesToPause;
                if(mod == 0 && !self.stop) {
                    $(self.options.appendTo).append("<div id='scroll_append_more' class='scroll_append_more'>"+self.options.moreText+"</div>");
                    self.pause = true;
                }

                // Put the footer back where it belongs if we've hit a pause
                if(self.pause || self.stop) self.setRelative();

                if(self.options.callback) self.options.callback.apply();

                self.loading = false;
            }
        });
    },

    // Saves the appended data to localStorage and handles the limit by clearing old data.

    saveData: function(key, value) {
        try {
            localStorage.setItem(key, value);
        } catch (e) {
            if(e.code == 22) {
                // localStorage Quota exceeded
                this.clearOld();
            }
        }
    },

    // Clears old cached data that has exceeded the given cache time limit

    clearOld: function() {
        for (var i = 0; i < localStorage.length; i++){
            var timestamp = Number(new Date());
            var regex = new RegExp("^time_(.+)", "g");
            var key = localStorage.key(i);
            var match = regex.exec(key);

            if(match) {
                var params = match[1];
                var prev_timestamp = localStorage.getItem(key);
                var diff = timestamp - prev_timestamp;
                diff = Math.round(diff/1000/60);

                //console.log("Age of cache: "+diff+" mins");
                if(diff >= this.options.expireCacheMins) {
                    localStorage.removeItem("scroll_"+params);
                    localStorage.removeItem("p_"+params);
                    localStorage.removeItem("time_"+params);
                    //console.log("Expiring");
                }
            }   
        }
    }

});
})(jQuery);

Comments

Sign in to comment.
dodo_mi   -  Mar 25, 2014

first of all thanks a lot for your great job!
I need the page refresh everytime I call it so I tried 2 ways:
adding a random number in the params option;
setting disableCache = true.
I noted a strange behaviour using Safari: in both cases I cannot go back to the exact position of the list.
example at http://www.sgpitalia.com
could You please help me?

Hawkee  -  Mar 25, 2014

If you disable the cache it won't go back to the same place. The cache is what allows you to go back to the same place.

dodo_mi  -  Mar 25, 2014

thanks for your quick response!
I know it but with other browsers I've no problem. in any case I need that a page is refreshed every time it's called: you can see I created a child function "clearOldHome" that clear cache without checking expireCacheMins. my problem is I don't know when I've to call it..

Hawkee  -  Mar 25, 2014

Why would you need to clear the cache outside of the expireCacheMins window? You could just set that value lower. The problem with disabling or clearing the cache is pressing the back button to get back to the page will always start it at the beginning.

dodo_mi  -  Mar 28, 2014

hawkee, thanks a lot!
Is there a way (method) to call the append command manually?
I've tried triggering the #scroll_append_more button ($('#scroll_append_more').trigger('click') or $('#scroll_append_more').click()) but nothing happens.. any suggests?

Hawkee  -  Mar 28, 2014

Why would you want to append manually? I didn't provide a mechanism for that.

Sign in to comment

jkurzner   -  May 06, 2013

First, thanks for a really cool plugin. I have gotten the scrolling to work great, but I had a couple of questions. First, my More button never shows up when I scroll down. Only a text version. I included all the required libraries, but perhaps there is a style missing?

Second, at the end of my list the word "false" shows up in my result set since that is what is supposed to signal the end of the data? Is there a way to remove that?

It also will show up multiple times if I hit the show more text link? Perhaps the two situations are related? Any guidance you could provide a newbie would be much appreciated.

Hawkee  -  May 06, 2013

You'll have to style the more button yourself. There is some example code above to help you style it. It should not show the word false. Is there any whitespace or any other characters that might be interfering?

jkurzner  -  May 06, 2013

Nope, I checked the page that feeds the scroller and the only thing on it is the word false when there are no additional records.

Here's a sample link http://sfuysa.com/playerprofiles.asp

I'll take a look at the more button, but I did include the styles shown above?

Hawkee  -  May 06, 2013

Even a single space before or after the word false will cause this problem. Make sure you don't have any other scripts throwing something into the header.

Hawkee  -  May 06, 2013

I just confirmed this on your sample link. There is a space after the word false.

jkurzner  -  May 07, 2013

You are so right and I am so embarrassed. Thanks so much!

Sign in to comment

ProIcons   -  Mar 13, 2013

I can't make it work on my PopOver content,

                            var notifas=10;
                            $(notif_scroll=function() {

                                $('#notif_drop').scrollAppend({
                                    url: 'notifications.php',
                                    pixelBuffer:50,
                                    params: { action: "more", start: notifas},
                                    appendTo: "#notif_drop",
                                    footerClass: "#footer_drop",
                                    callback: function() {
                                        console.log("start:"+(notifas));
                                        notifas+=10;
                                        $('[rel=timeago]').timeago();
                                        notif_scroll();
                                        $(".popover-content").mCustomScrollbar("update");
                                        $(".popover-content").mCustomScrollbar("scrollTo","top");
                                    }
                                });
                            });
                <div id="notif_drop">
                <ul class="not"><li class="notif">
                        <a class="notif" href="/core/feed.php?id=104&to=/about">
                            <img src="/img/default_user_icon.jpg">
                            <span class="header">
                                <span class="name">_Minimal_</span> 
                                <span class="desc">eafcjsay</span>
                            </span>
                            <span class="time"><i class="icon-time"></i><abbr title="2013-03-13 15:29:13" rel="timeago">about 8 hours ago</abbr></span>
                        </a>
                    </li>
                </ul>
                .................................................................................................................................
                <ul class="not"><li class="notif">
                        <a class="notif" href="/core/feed.php?id=63&to=/about">
                            <img src="/img/default_user_icon.jpg">
                            <span class="header">
                                <span class="name">NetM</span> 
                                <span class="desc">rjwrwfum</span>
                            </span>
                            <span class="time"><i class="icon-time"></i><abbr title="2013-03-13 13:53:21" rel="timeago">about 9 hours ago</abbr></span>
                        </a>
                    </li>
                </ul></div>
                <<div id="footer_drop">End</div>

but when i scroll at the end of the page, it load the content inside the popover... :/

Edit: I also changed some lines into the source code

Line: 206
$(self.options.footerClass).addClass('footer_fixed')
->
//$(self.options.footerClass).addClass('footer_fixed')

Line: 212
self.position = 'fixed';
->
//self.position = 'fixed';

ProIcons  -  Mar 13, 2013

Well i Noticed in the code that is only for WINDOW, can you make it dynamic for each content or shall i make it?

Hawkee  -  Mar 13, 2013

Yes, this only works for the window and not multiple elements on the page. Feel free to edit it to work for multiple elements. If your change is backward compatible and elegant feel free to push it to the github repo.

Sign in to comment

ProIcons   -  Mar 13, 2013

Excellent Job gj:D

 Respond  
mitke013   -  Mar 08, 2013

Great plugin! I have a question: how do I restart the plugin? Or reset page value to 0 so further scrolling will call ?page=1, ?page=2 ... instead of continuing from last remembered value?

Hawkee  -  Mar 08, 2013

Just set disableCache to true in the options.

mitke013  -  Mar 09, 2013

It is not about cache or page reload, I am still on same page. Example:
I have newsfeed that is loaded using your plugin (works perfectly). Let's presume 5 pages were loaded that way.
User fills in the form just like on facebook, all via ajax. Form passes validation and now, newsfeed loads itself using ?page=1 (ajax). But if I continue scrolling, plugin will continue from last remembered page, in this example 5.
I know I should just append that new post on top but for some other reason I cannot do that (because of applied filters, sorting etc.. it is simpler just to refresh).
Sorry for bad english.

Hawkee  -  Mar 13, 2013

You might need to write a sister function to clearOld that forces a cache refresh.

Sign in to comment

mfoda19   -  Jan 03, 2013

I tried doing that with .remove() but the issue is that every time remove is called everything gets shifted up to fill the gap left by the removed items, which looks pretty confusing and jerky. Is there another way to do this?

Hawkee  -  Jan 03, 2013

Hmm, that's a good question and probably why this sort of functionality doesn't exist right now.

Sign in to comment

Hawkee   -  Dec 26, 2012

@mfoda19 I'd just remove the elements from the DOM tree. jQuery makes this quite simple. The only tradeoff is it would remove content that could be searched within the browser window. This would be ideal for displaying pages of images.

 Respond  
mfoda19   -  Dec 26, 2012

That would be the most ideal solution. Come to think of it though, how would you handle the first part, removing old content? Wouldn't viewable content shift up every time you remove old content, since that space would become empty?

 Respond  
Hawkee   -  Dec 25, 2012

@mfoda19 That's a good suggestion. I could even take it a step further and reload the removed sections if you do want to scroll all the way back up.

 Respond  
mfoda19   -  Dec 25, 2012

Really cool plugin. Would it be possible to add the option to start removing old content after a certain number of page loads? For one thing when users scroll past content they've already viewed there's a very small chance that they would scroll back up to view it again. This becomes truer the further away they scroll from it. The real issue though is that loading content this way uses a lot of memory, which means after a point the scrolling will become a lot less smooth and the user's browser could end up crashing. More damage done to user experience than convenience gained from infinite scrolling, wouldn't you say?

 Respond  
Hawkee   -  Dec 22, 2012

Wow, over 1,200 views to this page in the past day thanks to Hacker News and Twitter.

 Respond  
Hawkee   -  Dec 22, 2012

@Apathetic It appears when you reach the initial bottom of the page.

 Respond  
Apathetic   -  Dec 22, 2012

Ah, I saw it! Though it's kinda hard making it appear.

 Respond  
Hawkee   -  Dec 22, 2012

@Apathetic Thank you. What browser and version are you using? If you scroll too fast you might miss it.

 Respond  
Apathetic   -  Dec 22, 2012

On the demo, I don't see the footer floating.

 Respond  
Hawkee   -  Dec 21, 2012

I just added a new feature that fixes the footer rather than pushes it out of view. It will remain fixed for one page of results then disappear until a pause is reached.

rogeraberg  -  Nov 27, 2013

Great work! How do I feed a startpage from the url? If I have site.com/?p=10 I wanna start att page 10, but the script always starts at 1. Thanks!

Sign in to comment

lmonfett   -  Nov 30, 2012

Got it. Thanks!

 Respond  
Hawkee   -  Nov 30, 2012

@lmonfett Just go to the Homepage here or the Explore section. Those both use this.

 Respond  
lmonfett   -  Nov 30, 2012

Thanks for this! Any chance you could set up a demo page so we can see some example code? Or can someone who's managed to implement it share their link? I'm still struggling to get it to work.

 Respond  
gpallis   -  Nov 23, 2012

Awesome, thanks! disableCache seems like exactly what I'm looking for. Thanks for a great plugin.

 Respond  
Hawkee   -  Nov 22, 2012

I added a couple new features:

  1. When the first page loads and it doesn't fill the screen the second page will be automatically appended.

  2. Added new disableCache option to start from the first page with each page refresh.
 Respond  
Hawkee   -  Nov 21, 2012

Yes, this is correct. The page number is stored because the information that was appended is also saved. So going back to page 1 would create redundancies. It will resume where you left off at the place you left off. This is so you can scroll down several pages, leave the site and return to your place without needing to scroll down several pages again. If the data is changing very quickly you can change expireCacheMins to a lower number so the cache is cleared more frequently.

 Respond  
gpallis   -  Nov 21, 2012

Hi Hawkee,
It's very elegant! The issue I'm having is that if I visit the site, scroll down a little, then refresh the page or return after browsing another site, the values sent as 'page number' don't reset to one, but rather stay at whatever they were at during my last visit. The line I'm commenting out is "self.page = localStorage.getItem('p_'+self.param_key);" - it seems from this time that the behaviour I'm describing is intentional, so I'm trying to get my head around what advantages it offers! My use case is fairly typical - a leaderboard that the user can scroll down.

 Respond  
Hawkee   -  Nov 20, 2012

@gpallis pageVar is just the variable name that your server-side script accepts as the page number. The script will increment it one by one as you scroll requested page after page of information. What line number are you commenting out?

 Respond  
gpallis   -  Nov 20, 2012

I was able to get everything working by commenting out the line that gets self.page from localStorage, but I'm sure that shouldn't be necessary!

 Respond  
gpallis   -  Nov 20, 2012

I'm having some issues using pageVar. The number it outputs seems to increase consistently, preserving state across refreshes and even browser cache clears. Is this the intended behaviour? I've only tested using Chrome.

 Respond  
Hawkee   -  Apr 18, 2012

@gooshie You'll reach the bottom after three appends.

 Respond  
gooshie   -  Apr 18, 2012

LoL there seems to be no bottom to the main page now!

 Respond  
Jordyk19   -  Apr 16, 2012

Excuse me it was 3 indeed.

 Respond  
Hawkee   -  Apr 16, 2012

@Fawkes Yes, exactly.

@Jordyk19 I've got it set to only 3 appends before it pauses here at Hawkee.

 Respond  
Are you sure you want to unfollow this person?
Are you sure you want to delete this?
Click "Unsubscribe" to stop receiving notices pertaining to this post.
Click "Subscribe" to resume notices pertaining to this post.