Local development made easy
Local development should be easy. For developing a new feature, we should focus only on code, and forget all about external dependencies.
Each external dependency is an additional, often unnecessary, effort in the development process. The more time it takes to set up the development environment, the less effective we are. And if we, on top of that, need to take care of our in-progress changes affecting others, we definitely will not be as productive as we could be by just focusing on our code.
Local environment ideally would simply work after we clone the repository and run it. Of course, this is not always possible, but we should strive towards it. We can come close to this “ideal” working conditions by mocking as much as possible.
Database
First thing that comes to mind is the database. Using a shared database for the whole development team complicates things a lot. Adding new features that do not touch existing tables should not be the problem. But if we are altering existing tables, we are effectively breaking the development environment for all other members of the team.
Testing different solutions for the same problem becomes more about how we can rollback changes if we choose not to go that way, than implementing that feature. Or even worse, we choose the first thing that comes to our mind and just go with that, to avoid the effort of rolling back database changes while keeping the solution stable.
If we choose to use a local database, all these problems go away. Now we will need to implement a seeder to fill our database with meaningful data. Arranging and maintaining this will take some effort. But it gives us an opportunity to analyze data and better understand our system. And we do not need to do it all manually. There are tools that can generate fake data that make sense.
This can be done directly in the database, but I like to keep it in my solution, and use libraries like Bogus, Faker.js or FakerPHP. This way we can determine when and how data will be seeded.
We all worked with a system that needs to send emails. This is often done by having a set of predetermined emails that you can use for testing. In most cases this is an OK approach. Although it is a little bit annoying that we need to switch between different accounts to check if an email was received. Or we need to configure the server to redirect all emails to one account.
Once set up, it is not that much of the effort, but it still requires extra attention. We can avoid that and simply use a dummy SMTP server like smtp4dev. Messages received in smtp4dev can be viewed and inspected locally, instead of login on a remote system.
File upload/download
In modern development we are focused on developing solutions for the cloud. This has changed how we handle files. We used to upload files to the server, and our API would save it to local or network disk. Now it is more common that our solution uses some services for storing files, like AWS S3 bucket.
So, each environment in that case will have its own bucket. That often includes the local development too. But there is a better solution. LocalStack enables us to emulate popular AWS services locally.
This is great for AWS, but not everybody is using AWS. No matter what service we are using, with little “trickery” and well-established abstraction we can solve this problem too. We can have two different implementations: one for environments, and one for local development.
The one for local development can simply store and read files from our local disk. We should not care about the source of our files, only how we can use it to implement new functionality. And a lot of our problems regarding cloud storage, access and/or rights are avoided this way.
We do not need to have access to infrastructure to be able to implement things. Our development process should focus on programming only, but a lot of companies do not see it this way. They do not see a clear separation of responsibilities between developers and devops, and expect developers to use and tweak infrastructure on a daily basis. Don’t get me wrong, we should know how it works and how to use it, but not necessarily explicitly use it.
External APIs
These days there are really small amounts of software that do not integrate with some external API. System we are trying to integrate with could provide us with a test instance. But that is not always the case. And even if we do have a test instance, we may choose to use it only for our environment, because there is a rate limit, or single account available.
In this case we can reproduce behavior or that external API by creating a mock server. Postman has the ability to create a mock server from collection, and that collection can be saved alongside our solution in our repository or on a team account.
We can also use mock libraries like WireMock.Net in which case we reuse those same mocks in integration tests.
No matter which way we decide to go, by eliminating external dependency we are no longer dependent on it and can focus on our development, regardless of limits that API can have. By controlling response, we are able to test all possible scenarios locally, even those hard to reproduce, and implement adequate logic for it.
Message queue
If we are working with event driven architecture, or simply want to delegate some work to out-of-process tasks we are probably using some version of a message queue. If we are using RabbitMQ or Kafka we can easily run docker images locally. If we are using AWS SQS we can use LocalStack as mentioned above. If we are condemned to use Azure Service Bus, we are probably using real subscriptions or topics, and adding the suffix “local” or developer name.
But we should really look at what we need for local development in this case. For receiving messages, we are using some library, or we are using our custom implementation. In both cases this should already be covered with tests either by us or by the creator of the library. So, message publishing and receiving should be outside of our scope.
What is in our scope, and what we should focus on, is the processing of the received message. We do not care how we received it and what the source of that message is. And since we do not care about the source, that source does not necessarily need to be the same as in production.
With good abstraction we can replace real consumers with a simpler implementation like database or mock endpoint. Postman can create this kind of mock server. Or if we use a library for this, and it supports multiple different implementations, we could simply switch to RabbitMQ or some other implementation that can be run locally.
We can have configuration parameters that will determine which implementation will be used. For the local environment we can set it to mock implementation by default.
This way there is no need for extra effort. No need to create subscriptions or topics only for local development. There is no possibility that we forget about those subscriptions or topics, and accidentally fill subscription storage. There is also no possibility that we trigger something that should not be triggered because we forgot to change our consumer/publisher to use a topic for our local development. And all of this can work offline, without needing to have access to a given resource.
Conclusion
I worked on many different projects, but when I have an opportunity to work on a greenfield project, I tend to make development on that project as autonomous as possible. One of my fondest memories in my career is the moment when a junior came to the project I set up. The same day when he arrived, at a daily meeting, the project manager asked him if he needed any help setting up the project. To this the junior replied that everything is already set up and he can start working immediately.
The reason for this was explained in this article. Project was configured to set up an environment if it was run with dev configuration. If the database did not exist, it would create a new one locally and execute all migrations.
After that seeder would check if there was data in the database, and if tables were empty it would generate some random, but credible, data.
For messaging, we used AWS SQS, but if run locally it would be replaced with an in-memory queue. This was a smaller project, so it was implemented as a monolith. So, in this case in-memory solution was good enough.
For emails there was an option for “demo” mode intended for local development. Regular implementation was wrapped with a decorator that would replace the real email address with a template that looked like test-mail+{original-mail}@project-domain.com. Email providers would ignore everything after plus sign and all emails were sent to the same email address. We could distinguish who was the intended receiver because the display name and part of the email address after + was filled with real data (for example John Doe <test-mail+john-doe@project-domain.com>).
Again, for local development instead of the AWS S3 bucket we used an implementation that stored and read data from a local disk.
These simple “tricks” enabled a junior developer to get onboard and start working on new functionalities within hours. Everything just worked on the first try. There was no unnecessary time wasted on introducing 100 dependencies and setting up accounts and rights. He did not need any of that to be productive. And that is a reason enough why we should care about the local development environment more.
