Denys – Linux, AWS, DevOps

We had an old issue with packer builds failing, overengineered logic with builds and inside AMIs.

• Refactored the build process, and got rid of all failing and over-engineered scripts;
• Prepared new simplified templates to build newer images;
• Found workarounds that allowed to build AMIs in china in a reasonable time.

Investigated Aurora performance issue, analysed application code and DB utilisation approach. Postgres is a core database for particular project, we faced a huge production performance impact for certain functionality.

Identified issues, worked with Dev team and Amazon representatives to figure out path forward. Finally moved DB from Aurora to Postgres and proposed a way to mitigate the issue in app code.

Application that connects doctors with patients and enables online interaction.

• Rewrote the Docker file to fit into the K8S world;
• Set up automated deployments with GitHub actions and documented everything;
• Created helm templates to simplify new env creation;
• Added integration with Teams chat.

The goal was to spin up an Apache Spark cluster that could be shared between different departments. One of the most challenging problems was ensuring the cluster scales up properly - ensuring that the scheduler and scaler interact properly and Spark jobs are scheduled, considering driver/worker relations.

• Installed Spark operator;
• Made sure cluster auto-scales properly;
• Tested solutions;
• Customized image build process;
• Verified scheduling works properly.

Jenkins build, which applies to Terraform code on merge to all VPCs. Previously applying was done manually, now it is with a fully automated process. After the merge, all VPCs wpould be in a consistent state in terms of security groups.

• Took broken Terraform code and went through all workspaces, recovering broken states, moving parts between Terraform modules, adding new sets of rules, conditions, and so on;
• Brought the code, after all the changes, to state when it could be applied safely to all VPCs (including production);
• As of now, Jenkins checks out code after merge and runs terraform apply on all workspaces.

It was a problem with China blocking traffic, which led to making a workaround in a rush and then working with all the customers to migrate them to a permanent solution.

• Had to devise a workaround as an emergency;
• Rolled out a workaround for several services;
• Had to start the long process of talking to the customers who had domains hardcoded in their systems.
• Had been talking with each customer helping to discover all integration nuances;
• Planned and executed migration from workaround to permanent solution.

A startup that gives the end users a fully functional Windows desktop in the browser window. Everything is remote - windows is real but runs somewhere, end user uses a browser instead of standard terminal utilities to get a desktop. That was probably one of the funniest projects - the startup owner argued with his tech team, and he ended up with all AWS instances deleted (except for a few QA machines).

• Got access to AWS, 2 QA machines without any services running, some stuff in S3, access to repositories in Bitbucket, a huge PDF document with several years of technical Skype chats, and a suspicious and frustrated business owner;
• Got the service up and running - website, desktop streaming, etc.

Payment system, processing payments from end-users.

Denys was sysadmin lead with 2 people in his team. They've built whole infra from scratch - bought dedicated servers, provisioned systems for OpenVZ, provisioned containers for each server role. Prepared dev/stg/prod environments and made CI builds for all different purposes (tests, deployments, etc).

The project was fully prepared to go live - monitoring, backup, provisioning, documentation, performance tests, and disaster recovery tests.