Job Dispatch is a personalized jobs newsletter that sends you new jobs from companies you follow every week. It's a tool I've been building together with @bzaalig over the last 3 months.

I wanted to write down the experience developing Job Dispatch using Webflow, Nodejs, Firestore, Firebase functions and storage, Sendgrid and a few 3rd party APIs like Lever and Greenhouse.

I want these and the following posts to give some insight into how actual projects get built, rather then this becoming a technical tutorial to setting up Firebase and Sendgrid.

What is Job Dispatch?

We built Job Dispatch because we saw 2 distinct problems with job hunting:

1. It's difficult to discover interesting companies we would want to work for. Job boards felt diluted and LinkedIn had a lots of noise and low signal.
2. Once we did find an interesting company, they oftentimes didn't have ways for us to stay up to date, which meant we found ourselves checking the career site frequently to check for new jobs.

We wanted to flip the job-search script: instead of candidates having to look for jobs, we wanted the jobs to come to candidates; to get notified when something interesting would pop up.

Tech stack & overview

Technically, Job Dispatch is 3 moving parts:

1. A chrome plugin that identifies if you're browsing a career website, and if so allows you to subscribe to it (solving problem #1)
2. A backend that fetches new jobs and creates personalized newsletters (the bulk of the logic lies here) (solving problem #2)
3. A webflow website that allows us to curate companies and create collections for users that they can follow (solving problem #1)

So, how does it work?

A frontend – there can be multiple –  sends a simple JSON payload to a RESTful endpoint, which is an express app built on Firebase functions: { email: "hello@toonverbeek.com", companies: {"companyA: "greenhouse", companyB: "lever"} }

This endpoint puts the backend to work to check if a user exists with this e-mail and if not, creates it and immediately subscribes the user to the supplied companies.

Depending on what happens, two distinct events can get triggered: user created or user updated. If user does exists, the backend creates a subscription request. This is an important part in keeping the experience password-less. It requires the user to confirm their subscription every time they add new companies to their dispatch. If a user doesn't exist, the user created event triggers a background function which sends the user a confirmation request e-mail.

When the user clicks this confirmation link the backend updates the user and either marks them as confirmed: true or updates the list of companies this user is subscribed to.

Then, once a user is subscribed they will receive a set of e-mails on a schedule. Every week on Friday, a Firebase scheduled function (a cronjob, essentially) is triggered that leverages Greenhouse and Lever APIs to find new jobs. It then generates a piece of HTML that is the unique set of jobs for a specific user. And finally, once all the dispatches are generated, I manually trigger a function that sends out all the e-mails through Sendgrid.

That's the whole flow! Now let's dive into the tech stack.

Requirements & picking a tech stack: the Firebase suite

Choosing a tech stack was relatively straightforward. First, we wrote down four requirements:

1. The user should be able to subscribe to 1 or more companies using either the Chrome extension or from the jobdispatch.co onboarding page
2. The user should be able to confirm their e-mail address
3. The user should receive a weekly e-mail every Friday with all the new jobs from the companies they're subscribed to
4. The user should be able to unsubscribe from any or all e-mails

Here's the actual scoping document I wrote back then:

Notion – The all-in-one workspace for your notes, tasks, wikis, and databases.

A new tool that blends your everyday work apps into one. It’s the all-in-one workspace for you and your team

Notion

From a product perspective, we also wanted to make the whole experience password-less and as easy to use as a regular newsletter. That meant that if a new user wanted to create 'an account', they should be able to do so from either the Webflow pages or the Chrome extension. I ended up implementing a rather naïve version of magic links for this (more on that in a different post).

The stack

As with all MVPs I've built before, I decided to stick with what I know best and utilize Firebase for the whole backend. These are all the services I'm using:

• Firestore as document store
• Firebase functions for a serverless backend. These include background triggers and scheduled functions (more on that later)
• Firebase Sendgrid extension that takes care of sending e-mails
• Firebase storage to store HTML

SQL or NoSQL?

The biggest trade-off I had to make was whether or not to use Firestore, because it being a document store rather than a SQL database, running queries is non-trivial. Barbara and I discussed early on that we wanted to make data the leading factor when deciding on which features to build. Going down the Firestore route meant that meant I would need to do extra work to extract data since I could not write SQL. In the end I chose the short-term gains here - development velocity - since our focus was on trying to validate our solution as fast as possible.

Choosing an e-mail provider

I also chose Sendgrid because that's the e-mail provider I was most familiar with. Now, 3 months into the project, I don't think I'll be picking Sendgrid again the next time around. I find their UI to be somewhat clunky and slow, as well as their SMTP API documentation confusing. I think I will check out Postmark next time I will need to build anything e-mail related.

Creating a production copy of Firestore for local development

As the project grew, so did my need for having a reliable testing setup on my local machine. One aspect of that meant having data that mimicked production as closely as possible. My first idea was to dump the firestore database and tweak it from there. I found a nice package called firebase-backup-restore (GH) that allowed me to do this. I created a little script that 1) could dump my production database and 2) import it to my local emulator setup.

Deploying to Firebase functions with Github Actions

One other thing that became relevant down the road was being consistent in how I deployed the project. I was deploying the code using the Firebase CLI, but at some point I started forgetting which branch / version was live. So I set up Github actions to deploy the project when anything got pushed to the master branch.

Next up: sending scheduled e-mails

In the next post, I go through how I set up Firebase to send transactional e-mails on a schedule.