the historious blog

kindle/ipad friendly links, referrals

These past few weeks have been very interesting here at historious, for various reasons. We have a few changes, so this is going to be a brief but important post.

Firstly, if you have a Kindle, an iPad or another mobile device, you can now view your historified articles much more efficiently by clicking the "mobilize" link below each result. historious will then use Google Mobilizer to remove most of the formatting from the target page, making it display much more nicely on mobile devices.

The second exciting feature is support for referrals! You've been asking for it, so we delivered. From now on, you will be able to refer your friends to historious and earn 100 extra historifications per friend (for each of you) for your free account, up to a maximum of 2,000.

To refer a friend, simply visit your settings page. There, you will find your personalised referral address. Simply give this to a friend, and when they sign up their limit will have been magically increased by 100 historifications, and so will yours. You can keep referring friends and watching your limit increase, all the way up to 2,000! This way, you don't have to subscribe, just introduce a friend to historious and you can both historify to your heart's content!

To make the process easier, we've included Twitter and Facebook buttons on your referral address. Simply click one of those buttons to post your referral code to your Facebook or Twitter page and start racking up those free historifications!

We hope you enjoy the new changes!

Team historious

service changes impact

You will remember that a few weeks ago we announced some service changes that would help historious deal with the amount of new users more efficiently, as well as some pricing changes. We wanted to give you a small update on the situation, as we don't like staying silent about matters that affect our users.

We are happy to report that the new limit for the new accounts has worked very well! We have had more signups in the days since the last announcement than ever before, and we handle increased demand beautifully. You will have noticed that the service is as fast as ever (Google reports page loading times of one second, but we're aiming for faster loads still) and everything works perfectly.

Our pricing changes were not as successful, as many of you felt that the new price was too high for the service. For this reason, the prices have been lowered to their previous levels (except that the yearly price is now a "pay for 10, get 12" deal). We would urge any customers who signed up for the higher rates to contact us and we will be happy to give them the new, discounted rates.

Another change we are planning soon is the removal of stale data from customers who no longer use historious but have not deleted their data. The way this will work is this: If a user doesn't use historious for a month (no logins and no historifications) they will get an email from us saying that their account has gone into an "unused" status and asking if we can do anything to help them. To reset the "unused" status, they can simply log into their account or historify something. After the account has been in the "unused" status for a month (i.e. the user hasn't logged in or historified anything for two months), we will consider the account to be abandoned and the data in it might be removed to make room for new users if necessary (unless the user starts using the account again in that period). We need to note here that subscribers' accounts are never considered abandoned, and abandoned accounts are considered used again on the first sign of activity from the user.

To clarify, since active users historify and search for things very frequently, they will most likely never get an email from us. The only time a user will get an email is if they haven't touched their account in a month. If you are planning to be away from your computer for a long time, you can email us and we'll be happy to keep your account fresh (or you can subscribe!).

The good news is that this will free up many of our resources and we will be able to implement many useful things for existing users, such as referrals (you will be able to raise the number of your free historifications by referring friends to historious) and sales of packages of historifications (e.g. buy 100 extra historifications for your free account with a one-off payment).

Overall, the new changes have reduced our operating costs by a very large factor, and we are happy to report that we now have many more resources available to accommodate a large increase in demand. Additionally, these changes are scalable, which means that current users will no longer have to bear the cost of users who have abandoned their accounts and historious should be much faster, more responsive and cheaper from now on.

We would like to thank you all for being patient through our exploration, and we are very glad to have been able to ensure that the service is now very much improved!

Thanks again, and happy historifying!
Team historious

decision-making and some service changes

When you see a successful company, it's only natural to think that they have some very competent people making decisions, who know what must be done at each step and know exactly how that will affect the company. While the former may be true, the latter is usually not. The way decisions are usually made is initially through guesswork (incorporating experience, best practices and common sense), and refining those guesses with data.

We are no different. There's no magic 8-ball that tells us which decision is the right one ahead of time (well, there is, but we aren't really sure it has any idea either). All the successful companies didn't know exactly what they were doing, too (then again, neither did all the unsuccessful companies you never hear about). All they had was a goal (we want to help people remember the things they see online) and an idea of how to go about achieving that goal (by doing what's best for our users). Everything else is just guesswork and doing what you think will benefit you and your users the most.

When we make a decision, we don't know what it will result in. We try to base our decisions on hard data, but that's not always possible. Much of the time, we go by what feels right and what the circumstances dictate.

In order to try and improve the service for everyone, we've made a few changes. You might have noticed that the limit for free historifications is now 300, and that the subscription price has gone up. Unfortunately, it turns out that keeping a cached version of every page is costly, so we would never be able to sustain the old limit for free accounts. We don't believe that taking away the cached page feature from free accounts is fair or beneficial, so we have opted not to do that. The price increase is part of a test to help arrive at a better price point.

If you created an account before the changes, you will notice that your limit is still the old one. Since we don't think that it's fair for us to take away what we had given, your limit will stay what it was. If you are a subscriber, the subscription price will also remain constant for as long as you are subscribed. If you were thinking of subscribing and think the current price is too high, just email us and we'll be glad to offer you a subscription for the old price, if you've signed up before today.

We are very sorry that we can't offer the service for free to everyone. Unfortunately, we have various costs ourselves, so we need the revenue. The current user:subscriber ratio is 100:1, which means that, for every subscriber with 3,000 historified sites, there are 100 people with 300,000 bookmarks (that's a maximum, granted) who do not pay anything. This is not something we can sustain indefinitely (we've already gone through two server upgrades to cope with increased demand), so we decided that the limit and pricing changes were necessary.

Hopefully the new limit will still be very useful to casual users, whereas power users with many bookmarks to import can subscribe and take advantage of the unlimited package.

Please let us know what you think of the changes here in the comments, on Twitter (to @historrific) or by email!

Thank you all for your patience and continued support, and we hope you will continue to find historious useful!

Happy historifying!
Team historious

on pricing

Update: Please help us with our pricing research by filling out our five-question survey!

Lately, we've seen many new, subscription-based startups wondering how to price their products, what plans to offer and how (if at all) to offer a free plan. Since we have a few months on experience on the issue, we'd like to share our thoughts. As you may know, historious has three pricing plans (which are really only two in disguise, as one is a discounted yearly plan and the other a regular monthly plan). The free plan has a few limits, such as the ability to historify up to 3,000 sites, at the time of this writing, and the paid plans are unlimited.

Please note that all the following is based on our experience. Any numbers, information or advice may not apply to you, so take it with a grain of salt.

First of all, it's no secret that a free plan is a great attractor of users. While a free plan doesn't (directly) bring revenue to the company, free users who like the service can recommend it to others, who might, in turn, buy a paid plan, or the free users might "graduate" into a paid plan themselves. Free plans are, therefore, great advertising for the company. There are two questions that stem from the previous sentence, and those are:

1. How likely is a free user to eventually purchase a free plan?
2. What is a ratio of free:paid users a company can reasonably expect?

To answer the first question, in our experience, not terribly likely. From what we've seen, paid users will usually purchase their plans within a few days of use of the service. Most people upgrade because they like the service so much that they want to support it, while others upgrade for the features. It is more rarely that a user will use a free plan for some time and then decide to upgrade, but that obviously happens too.

The second question is a bit trickier, as there's no clear definition of "user". In our case, when counting all the registered users, the ratio is about 100:1. That means that about 1% of your users will decide to purchase your service in the end. When counting active users (users who regularly use the site) this figure is a bit more optimistic, at 5%.

Knowing the previous information, the decision on how to price your product hinges on one consideration: Weighing the cost of free users versus the benefit. The benefit (referrals) is very hard to measure effectively, unless you have a referral program in place (which is a very good idea). If you monetize your free users, the benefit becomes easier to measure, and you can adjust your free plan higher if you are able to generate revenue from free users.

Here at historious, free users cost us quite a bit in terms of disk space. Since we cache each page, a free user with 3,000 links will take up around 60 megabytes of space, which is a huge amount for one user. This means that, unfortunately, we need to work harder to monetize the free plans, and we cannot offer too much leeway for our free users. This problem is exacerbated by the fact that users who decide not to use the service leave their data behind, so even users who are no longer with us cost money. Most applications will presumably see their free users not cost so much, and thus will be able to make their free plan more lenient. One way to deal with the "stale data" problem is to prune old users from the system, but many people will be understandably dissatisfied with this, and it is not recommended.

To recap:

1. Free users are not very likely to upgrade to paid, so severely restricting the free plan might not get you more subscriptions and might lose you free word-of-mouth publicity.
2. If you can monetize free users, do it. If free users generate enough revenue for you, you might not even need subscriptions.
3. If free users cost too much, you can restrict the free plan more. Users who feel too restricted will usually leave rather than pay.

Many of our users offer us their opinions on our pricing. Many users say that the limit is too low, or that the price is too high, or that a specific feature would be more useful if it were available in the free plan. We love all our users, free or paying, and would like to offer them unlimited plans for free, but the reality is that, without the revenue that paid users provide, the service would not exist. You need to remember that you will not always be able to please your users, and you might feel bad (at least we do), but it's necessary for both your and their benefit (nobody wins if you go out of business).

We hope you found this article useful, and happy historifying!
Team historious

why we were wrong about why we were wrong

A few days ago, we wrote an article titled how we improved signups by 30% by doing nothing (the oldest among you might still remember it). In it, we detailed our trials and tribulations on trying to perform A/B testing to improve our metrics on historious, and how we got statistically significant results (or, well, what we thought were statistically significant results). After some fantastic help from the community (especially Hacker News), we reached some new conclusions on why we got the results we got, and what went wrong.

As you might remember if you read the article (summary: we got a 30% improvement on signups with 99.8% confidence on a change that didn't actually exist), we got a 30% improvement on signups with 99.8% confidence on a change that didn't actually exist. This shook our confidence on the confidence measure somewhat, as we saw that the formula was extremely confident about its performance, even though it was abysmal, and thus was only fit for an audition in American Idol.

What many of you pointed out, however, was that the error lay in our methodology. As it turns out, you can't just obsessively monitor your A/B test results all day and decide to stop when confidence is over a certain threshold. This is because each time you look at the test results, you sample that distribution, so if you aim for 95% confidence and check ten times, you're more likely to get that reported confidence one of those times, by pure chance. If you only decide when to stop the test by the reported confidence level, then the test becomes moot. This is the statistical equivalent of a watched pot. It won't boil.

How can we guard against this, though? The answer is simple: Decide on your sample size beforehand, and only stop the test after you've reached it. This way, you won't be taken in by insufficient data, since you'll be disregarding any values before your test sample size has been reached. For more information, see How not to run an A/B test. The required sample size depends on a few factors, such as the size of the result you expect to see and the expected variance.

There are a few more techniques one can use to decide when to stop the experiment, but they are beyond the scope of this post. The best thing to do, if your sample size has been reached but you aren't sure if you should stop your experiment, is probably to let it run a bit longer and gather more data. Also, please keep in mind that, given our track record, everything we've written here is probably wrong.

Once again, thank you all for your comments and help, and we hope you keep our experience in mind when performing your own A/B tests!

Happy historifying!
Team historious

how we improved signups by 30% by doing nothing

You've (hopefully) heard many things about A/B testing. People rave about it, and with good reason: It provides you with a solid framework on which you can evaluate how a change you've made has affected your metrics. Having science backgrounds, we're big believers in metrics here at historious, and we think that any business decision should be based, at least in part, on those metrics.

In case you're unfamiliar with A/B testing, here's a small rundown (it might get a bit technical, but please follow through):

You have two different versions of a page, one with a change and one with no change. You want to see if you should actually make that change, so you send half your visitors on the first page and half on the second. You monitor how many of the visitors on each page perform an action (say, sign up for your service), and then you have the signup rate for the first page (the changed one) and the rate for the second page (the unchanged one). Whichever page has the highest signup rate is the one you need to use.

How can you rule out randomness, however? If you show one person the first page and he signs up, and show one person the second page and he doesn't sign up, does this mean that the first page is better? Clearly not, since it might be pure luck. Therefore, we need a way to tell whether the difference in rate is statistically significant or not.

For this reason, we have a special formula that can tell us if the change actually is statistically significant. You enter the people who signed up on the first variation, how many visitors saw it in total, and the same for the second variation, then magic happens and you get a percentage that tells you how likely the change you have is actually statistically significant. If you get, for example, 90%, it means that you have a one in ten chance that the change was due to luck.

Since we wanted to minimise the chance of changes making our user experience worse, we decided to go for over 98%. This way, only once in fifty times were we supposed to get random results.

We made various changes this way, e.g. decide whether or not to include a presentation of the site's functionality or not (we decided against it, as it lowered the signup rate), the order of the calls to action, etc etc. After a month or so of running A/B tests, we decided to test the A/B testing software itself, to verify that everything was actually correct.

To do this, we had two versions of the page that were exactly the same. This should produce no difference in signups, and thus no confidence in the result.

We left this test running for three days, and then came back to see the results. A few thousand visitors had entered the test, and the results were clear: The "changed" page had improved signups by a whole 30%, with 99.8% confidence!

This was an astounding result! We had effectively made visitors 30% more likely to sign up by doing nothing!

It doesn't make any sense, however, so we started checking everything. Whether the second page actually had a change we hadn't noticed, whether the software biased visitors toward a page, whether there was a bug in the calculation code, and came up with nothing.

Everything was correct, yet, with 99.8% certainty, the variation was better than the original by a lot! Since there's no reasonable explanation for this, we decided to keep the test running and see if the trend continued.

Keep in mind that, with 99.8% confidence, we were well past our certainty threshold, and, were the two pages not the same, we would have definitely used the variation. Now, though, we didn't know what to think. Had our precious A/B testing methods been lying to us all along? Had we made some grave mistake? We searched high and low, asked on forums for a possible mistake we could have made, but nothing. Surely we hadn't fallen on the one-in-five-hundred case?

Another three days of running the tests, rates had reverted much closer to 50%, and confidence was a meagre 10%. This, at least, let us heave a sigh of relief, secure in the knowledge that we hadn't just disproved the whole of statistics and mathematics.

How many decisions, though, did we make that were wrong? If 99.8% confidence wasn't enough, what about the changes we made based on 98% confidence?

The important lesson here, and the one you should take away from our experience, is this: Whenever you think you have enough data for the A/B test, get more! Sometimes, you will fall into that 0.1%, and your decision will be wrong, and might impact your metrics adversely, and you might never find out.

Generally, the less significant the change, the more data you are likely to need in order to gain a good degree of confidence. For small changes, you might get a confidence of over 95% at some point, but it would be wise to gather some more data, as that's only a one in twenty chance that the result is not statistically significant.

The good news is that, if the tests are independent of one another (i.e. the probability of a user being in test A is independent of the probability of the user being in test B), you can run multiple tests on the same page and still get good data. Of course, you might have biases in the actual test, e.g. the two tests might work very well together but very badly each on its own, which would skew the data. Generally, though, you should be able to run two tests at the same time. This is especially useful when you're starting out and don't have thousands of visitors each day, providing you with lots of data.

We hope we have given you more insight into what constitutes a good A/B test, so you can improve your process and not repeat our mistakes. Thanks for reading, and have fun!

how we got 100,000 visitors without noticing

If you run or have ever run a website, you would be sure than getting 30,000 visitors in a day would be something you'd definitely notice, wouldn't you?

As it so happens, we did get 30,000 visitors in a day, and we didn't notice!

A few weeks ago, we had a look at our analytics page to see what happened the previous day. Before going any further, you need to know that, apart from our Google Analytics page, we have an internal analytics package that tells us various things about historious, such as the number of registered users, the number of indexed documents, etc.

historious had been running smoothly, so you can imagine the surprise when the Google Analytics page showed all our daily visits at zero:

Looking more closely, we noticed that the data hadn't actually gone away, what happened was that the previous day's visitors had absolutely dwarfed the number of visitors for any other day, with more than 30,000 people visiting historious. When we expanded the analytics to include the current day, we saw that we already had 50,000 visitors by then, and there still more coming in!

"How on earth can you get that many people and not notice until a day later?", you're probably thinking. "Also, how on earth can you get that many people, period?", you are probably also thinking. We certainly were!

To find out where all these people were coming from, we had a look at the referring websites. Google reported that all of those visitors were coming from Google itself, from organic search results. How was that possible? Getting that popular in Google searches doesn't happen overnight, and it certainly doesn't happen at this volume!

The next logical step was to look at the keywords people used to come to the site. Would the magic keyword be "social bookmarking"? How about "amazing service"? "A bookmarking revolution", perhaps? Hoping for something doesn't make it true, however, and the reality was much more surprising: Apparently, the claim to fame for historious, the amazing functionality that made it an overnight sensation was... "free sms"?!

How could that be? historious doesn't offer any sms services, let alone a free one! This mystery was getting more and more mysterious. Luckily, the "popular pages" pane had the answer: it seemed like every one of those visitors visited a historious cache page.

If you aren't familiar with historious, it works by retrieving any page you bookmark and indexing it. However, along with indexing the page, historious saves a copy, so you always have the page available the way it was when you first bookmarked (we call it "historified") it, no matter how the original website changes (or disappears altogether). Since you can also make your historified sites public, you can also publicise these cached pages.

Apparently, one of our users had historified a Brazilian site that offered free sms, and Google had indexed it. By some algorithmic oddity, the historious cache actually ranked higher than the site (which turned out to be the most popular free sms service in Brazil), so anyone who searched for "free sms" in Brazil ended up on the historious page!

To top things off, since we take great care in making the cached pages functional, none of the user knew that they weren't on the site they thought they were (not many people seemed to notice the historious information pane on the top of the page). Even the SMS form worked, so all the visitors were content with this.

Answering the question "how didn't we notice?" is now easier. We didn't notice because not many of those visitors actually visited the main page of the service, and even fewer people signed up as a result! This is less than ideal from a business standpoint, but it showcases the potential historious has, and we are always happy when we provide such a good user experience that the user doesn't even realise that they were on historious!

"Surely, however, your server would exhibit some load with all these people?!" you ask. Well, no. You see, we have Varnish serving as a caching frontend for media and some pages, including cached pages. None of the requests ever hit Apache (or, indeed, got anywhere past Varnish), and Varnish is so efficient at serving pages that the site load remained at a cool 0.00(!) for the entire duration of the incident.

That duration turned out to be a little longer than three days, netting us more than 100,000 unique visitors in that time. Thanks to Varnish, none of our users ever noticed anything different, and the whole of Brazil managed to send their free SMS quickly and easily!

asynchronous processing using celery

As we posted earlier, we have converted our backend to work asynchronously. In this post, we will explain what this means and how one can go about writing an application that will use asynchronous queues with celery. If you aren't particularly technical, this post is unlikely to interest you very much, but you're still welcome to read on to get a glimpse of how the internals of historious work!

As you probably know, historious is a bookmark search engine. You enter a URL, it gets downloaded (unless you provide the source, which has many advantages such as bypassing paywalls and registrations) and indexed. This way, you can search across all your bookmarks for any word or words that appear in the text of the page, rather than just the title or the tags. You do also have the option to tag posts, but that is optional.

So, to support this architecture, we need something that will accept a URL, download it, index it and store it. The way this was done until a few days ago was this:

1. The URL is stored in a database table (along with the source, if it exists).
2. A daemon polls the database every second for documents that haven't been downloaded yet, downloads them and stores their source in the above database table.
3. A second daemon polls the database every second for documents that haven't been indexed yet, indexes them and marks them as indexed in the database.

While this worked rather well until now, it is a rather big waste of resources to have to query a table with millions of rows every second. The table is indexed, of course, so the query only takes a fraction of a second, but the database still gets hit, which can be bad when the server is already under load.

This approach, however, has the advantage that a user who adds a URL doesn't have to wait for the URL to be downloaded and indexed, which can take a few seconds. The user is assured that the document they just historified has been successfully added within a few milliseconds, which is the time it takes to insert a URL in the database. This is especially useful when the user migrates to historious from another bookmarking service and has many thousands of URLs to add.

Instead of querying the entire table every second, we decided to improve this process a bit by setting a flag in our redis store every time a document was added. This way, we could just poll the (much faster) redis key for changes instead of the database, and only hit the database when necessary. This approach did improve matters, but it was a bit unwieldy in that the flag might be dropped and reraised in odd ways, and some documents were left unindexed for a few seconds until another document was added.

An even better solution, and the one we have implemented now, is to use celery to run the function asynchronously. This way, we can just call the downloading function, which will, in turn, call the indexing function when it's done. Since we are already using redis (and RabbitMQ doesn't support priorities yet), we decided to use the redis store for celery (redis has "databases", which are basically namespaces for keys, so the celery database doesn't interfere with our caching one).

Adding celery to the project was very easy using the django-celery package. To use redis, all one needs to do is add the following lines to the settings.py file:

CARROT_BACKEND = "ghettoq.taproot.Redis"
BROKER_HOST = "localhost"
BROKER_PORT = 6379
BROKER_VHOST = "0"

CELERY_RESULT_BACKEND = "redis"
REDIS_HOST = "localhost"
REDIS_PORT = 6379
REDIS_DB = "1"

Adding both backends is necessary, as the first one is the messages backend (where celery stores the function calls) and the second is the results backend (where celery stores the return values).

After doing this, we can just call our functions in the normal celery way (myfunction.delay(<arguments>)), and celery will take care of everything for us.

At this point, the first piece of the puzzle is done. What's equally important, though, is to ensure that users who run big import jobs won't disturb users who use the bookmarklet or extensions. When using the bookmarklet, the source of the current page gets submitted to historious, so we don't need to download it ourselves. Since the source is already there, we can just index it, and we need to make sure that these indexing tasks run before the downloading tasks.

When a user adds two thousand bookmarks, all these are being added as "download" jobs. This means that we will have to wait for all two thousand bookmarks to be downloaded (and indexed) before anyone else can use historious. This is clearly unacceptable, so we need to find a way to prioritise indexing jobs over downloading jobs.

Fortunately, celery supports task priorities. RabbitMQ doesn't support them yet, but redis does, and they work fine (the docs are a bit hazy on this, but they do work). To specify a task's priority, all we need to do is add the argument to the decorator (@task(priority=0)) Taking advantage of this, we just set the indexing jobs to have a higher priority than the downloading jobs, and celery will automatically prioritise any indexing over all the downloading jobs, even if the indexing job is added last!

As you can see, adding celery support in any existing project is very simple. Just add the above configuration in your settings.py, declare your tasks, run the daemon and call them! Celery has been working very well, both in testing and production (for the few hours it has been there). Most importantly, the load on the server remains very low, even though users are constantly adding new documents. This is a great improvement on the old way of doing things, and we hope you have gotten enough of a taste of celery to use it in your own projects!

Good luck, and don't hesitate to comment on this post with your experiences!

Team historious

redesign, rss feeds, asynchronous backend

After the frenetic pace of the first few weeks, things have quieted down here at historious, and we managed to make a few long-needed upgrades. Some of them will be immediately noticeable to the users, some will be more subtle, but all were quite necessary!

• Our largest change, and the one we are most excited about, is the redesign of the website. You will have noticed that all pages (except for the search and results page) now have a much more modern look, and we are migrating the search and results pages to the new design as well, since the initial update was very well received!
• An equally important but less visible change is our move to celery for the backend. This means that your historified pages will now be indexed more quickly and you won't experience any extra load if historious is not historifying any pages.
• We have added RSS feeds for subscribers. These RSS feeds are available on the search page of your private (and public) historious, and they contain updates on the latest historified sites you've added recently, so you can add them to your browser or phone!

Expect more updates to come soon, as we discover the impact of these changes to the site!

If there are any problems with the look of any page, new historifications appearing, or the RSS feeds, please tell us about it!

Thank you and we hope you enjoy the changes!

Team historious

the customer is always right

As you will know if you’ve ever contacted us, we’re always trying to make our users happy here at historious. Every email we receive during the day gets a reply in less than five minutes, usually within thirty seconds, and we can see that people don’t expect this, and love it. They are very surprised when they email us about an issue and, a minute later, they receive an email informing them that the issue has been fixed. This is the great advantage of having the developers also doing support, as, since they have access to the codebase, they can fix any small bug and push to production within a few seconds.

When one communicates with people, however, there are a few things one needs to know to make the interaction go as smoothly as possible. We’ve all heard the old adage, “the customer is always right”. To us, it conjured images of frustrated support personnel trying to deal with unreasonable customers, but still having to grit their teeth to avoid losing a customer. The reality is different, and what we found surprised us.

The customer is always right.

It’s not a guideline. It’s a statement of fact. In the past month, and countless emails, tweets and messages later, there has not been a single instance where a user has emailed us and was wrong about something.

The site’s purpose is unclear in the copy on the front page? It’s our fault for not expressing it clearly enough. It’s not very obvious how to perform a specific task? We didn’t make the help link stand out enough. The API is confusing? We didn’t write the documentation well enough.

There was one specific email that was asking for some piece of functionality that was completely irrelevant to historious and had nothing to do with bookmarking or searching. After we asked the person to explain their idea better, it turned out that it was actually a brilliant idea which we implemented right away. We discovered that every user had good ideas and was very willing to help us improve the service, even going so far as to debug issues we were having!

In the end, what we’d like to say to other services is this: Listen to your users, because they know what they need, even if they don’t always know the best way to get it. Your job is to find the best way to solve your customer’s need in a general and useful way that fits your product.

To our users, we’d like to say this: Thank you for all your help and support, and we will continue to try to make a service you can love! If we don’t always succeed right away, bear with us, and we know that, in the end, you are always right!