Improving Puppeteer Performance

May 3, 2019

One of the things that stands out when using a headless browser (versus cURL or other simpler tools) is that it can be painfully slow. Some of the cost is unavoidable -- you'll have to start the browser, wait for it to initialize, and then proceed from there. This is even harder in platforms that offer functions-as-a-service as there's a warming phase plus the fact that you cannot cache any results!

Today, I'd like to go over how browserless can help you mitigate some of these slower operations, and in certain cases even cut your loading time in half.

Painless Puppeteer Debugging

April 18, 2019

Joel Griffith

One of the things that becomes obvious when working with puppeteer is how frustrating and time-consuming it can be to debug. Node's debugging tools can only do so much (you can't see the browser), and taking screenshots at every step only scales so far. By the time you've fixed the issue, you're back to removing all your debugging code, which also costs precious time.

browserless is dedicated to improving the development and debugging experience of headless browser work, including when your scripts are even running in production. Given that most developers will likely know how to use Chrome's developer toolset, we also don't want to re-invent the wheel as well. With those thoughts in mind, and after a few weeks of building, today we're happy to announce our new Live Debugger. See it in action below!

Usage-based accounts are live!

April 1, 2019

Joel Griffith

We're happy to announce that usage-based accounts on browserless are now live! Similar to our dedicated accounts, usage-based accounts come with all the API's that you now expect from the browserless service with the convenience of only paying for what you use. You can now sign-up on our main website here..

If you've been fighting to get Chrome running stable in all the serverless places, this product was built specifically for you! All of Chromes dependencies are there, including missing fonts and other packages, plus we pre-boot Chrome for you ensuring that your start times are near zero. We even support running in "headful" mode as well! Here's a chart of some workloads we've seen with this new pricing structure:

Type of session	Total Time	Cost
Capture a full-site screenshot with `/screenshot`	2 seconds running	$0.00016
Generate a 5-page PDF with `/pdf`	5 seconds running	$0.0004
Scrape 10 pages in parallel	30 seconds running (each)	$0.024
Run a unit-test with 10 specs in parallel	5 seconds (each)	$0.0004
Complex sign-up test of your web-app	25 minutes	$0.12
Complex interaction with 10 pages in parallel	25 minutes (each)	$1.20

Observations running more than 5 million headless sessions a week

March 13, 2019

Joel Griffith

You might recall a post in mid-2018 where we announced we had served over 2 million headless sessions. At the time that was quite a milestone for browserless, given that we're a self-bootstrapped company of one.

After that announcement, and it's arrival on the front page of Hacker News, we experienced some pretty spectacular growth. Now, at the time of this writing, we currently run nearly 5 million headless sessions a week. Given all of this growth and time passed, it's time once again for us to divulge what we've learned running a headless browser company at scale.

five million sessions a week amplify

A snapshot of our nginx amplify dashboard

So sit back, grab your favorite beverage, and see how you can improve your headless browser scripts.

Sanely debugging puppeteer and fixes to common issues

February 26, 2019

Joel Griffith

Ever take a page.screenshot call every-so-often in your code to see what the browser is doing? Or is console.log('Step #1') sprinkled throughout your code? While easy to use when you need some quick feedback, these methods of debugging remind me of troubleshooting JavaScript issues back in the days of Internet Explorer 5. Of course we've come a long way since then, and also have much much easier ways to see what's going on in your puppeteer code, including the "gold" standard of Chrome's developer tools. Getting it all running properly, however, requires a bit of setup work which is well worth it.

It's my hope that, at the end of this post, you'll have much better tools and methods for debugging some of the tougher problems in headless Chrome. And if not you'll at least know where to get started!

Phantom pain: the first year running headless Chrome in production

January 7, 2019

Joel Griffith

Step back with me, almost two years ago, when Google announced that Chrome would support a first-class headless mode. Even though it was only April it felt like Christmas had arrived early! After years of personally dealing with projects that attempted to automate and scale a web-browser (Selenium, phantom and so on), the timing couldn't have been better as I was in desperate need of a performant solution. As much as I wanted to believe that headless Chrome would solve all of our collective development woes, and it does solve a good chunk mind you, the shocking truth is that there's still quite a bit that needs to be done.

New globally available workers!

August 8, 2018

Joel Griffith

We're happy to announce that you can now launch browserless instances across the globe! With this comes two new load-balancers that can better handle routing your traffic as well. We're starting small, but will be looking to build our global infrastructure so that no matter where you are you'll be closer to your browserless workers. If you're anxious to use the service, but haven't yet due to location concerns, then let us know where you're at!

Things they don't teach you running a business by yourself

August 1, 2018

Joel Griffith

One of the most hotly talked about topics in tech right now is starting an indie business. What is an indie business? It's one where you, and maybe a co-founder, attempt to start a business with no investor funding or large external influence. It goes by many names: boot-strapped, solo-founded, self-funded, indie-hackers...etc... and it's pretty hot right now! Which is why we see a proliferation of sites like IndieHackers and posts on Hacker News (frequently making it the front-page). While I think it's an exciting venture to take part of, I don't think it's for everybody. There's a reason that folks seek employment over entrepreneurship, and we often forget these points when we see how "Joe Schmoe" went from an individual-contributor role to making $50k/mo with his side-hustle selling artisanal crayons out of his basement. Also, spoiler-alert, that kind of growth is an outlier, and not something to expect or bet on. A large amount of these run-away success stories are backed by years of failures and experience that catapulted them to where they are now.

So why talk about this? As the sole-developer, founder, and doer of all things at browserless; I wanted to take chance and paint a better picture of what's like to be independent in the technology world. The highs, the lows, and some things you should expect. Which brings me to my first point...

Observations running 2 million headless sessions

June 4, 2018

Joel Griffith

We're excited to announce that we've recently just crossed over 2 million sessions served! That's millions of screenshots generated, PDF's printed, and websites tested. We've done just about everything you can think of with a headless browser.

While we're excited to have achieved this milestone there's definitely been a lot of hiccups and issues we've seen along the way. Because of the amount of traffic we've gotten, we'd like to take some time and outline common best practices when running headless browsers (and puppeteer) in a production environment.

Browserless functions and usage-based accounts!

May 26, 2018

Joel Griffith

Usage-based accounts are now live! Read more about them here.

We're extremely excited to announce that we'll soon be entering a closed-beta for browserless functions and a new payment option: usage-based payment! We've listened to your comments and suggestions, and we hope these new features will give you flexibility on how to organize your headless-browser work and how to pay for it!

← Prev Next →