Starting a Project: Data Simulation

Sometimes, there is a need to generate data to test your code. While “testing in production” is a sweet spot to be, there are times when you need to “force” (stub) data that reflects certain business phenomena in production.

This does not mean “chaos engineering” but rather a finite set that can be ingested into a suite of integration tests to increase confidence in code change.

For example, creating a number of threads to generate load can be done trivially with Apache Bench (ab), I’m looking for a scenario builder with lots of traffic data. I’m referring to the “needle in the haystack” scenario builder.

I started with some ideas on how to create multiple “stubs” in a thread pool so that I can force concurrent payloads to hit a service to test. Using the Spring Framework and Spring Boot, this task became super simple. I ended up with the following:

{% highlight java linenos %} @Async(“myThreadPoolTaskExecutor”) public Object createPayload() { // return payload pojo } {% endhighlight %}

and the following TaskExecutor bean:

{% highlight java linenos %} @Bean(name = “myThreadPoolTaskExecutor”) public TaskExecutor threadPoolTaskExecutor() { ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor(); executor.setCorePoolSize(4); executor.setMaxPoolSize(4); executor.setThreadNamePrefix(“executor”); executor.initialize(); return executor; } {% endhighlight %}

Of course, this is only a start with the following potential test:

• “Transaction boundaries” defined in my code.
• Verify the “concurrency” setup.
• Create a set of scenarios (the needle in the haystack)

I’ll update more on this journey as I add more to the data generator project I have been creating.