Retrospective on my biggest personal project

Today I'd like to look back at my longest (and biggest) personal project and what I learned along the way. The project is now called Last Run (but was named Arena of Titans when we started). It's a board game adapted for web usage. It went live a few month ago and is available here. The project is also entirely Open Source under the AGPLv3 license with the code source hosted on gitlab. If you play and have feedback, let me know! We are on social media or you can leave a comment here.

I started the project in 2014 when I was completing my studies in Centrale Marseille, a French engineering school. At the time, we were 4 on the project: the creator of the game, myself and two other developers. This project was accepted as our final project assignment which was a great way to combine passion and usefulness. Now we are three: the creator of the game who does a bit of development but is mostly working on the gameplay, myself as the main developer and a new developer who participates from time to time.

The original version of the game was written in Java and AngularJS. I remember I looked at Apache Wicket and Vaadin to do everything in Java, including the frontend. This idea was influenced by a fellow student (but not teammate) who really liked Vaadin (and maybe still does). Eventually, after some tests, it became clear the project wasn't really fitting into this model (don't ask me exactly why, it was years ago) and I settled on AngularJS (for the best I think). At this time, the frontend did HTTP requests to the API because it was easier this way given what we knew and all players had to play on the same computer because it was easier for us and we didn't really know how to update other browsers even if they were connected to the same API. Since HTTP requests should be stateless, I stored the game state in Redis which is the only original technical decision that remained over the years. It was also the early days of Docker, which looked cool and could allow me to avoid installing Java on my server, so I initially deployed the app with it. It was a bit painful (mostly because I did it at the very end of the project so the examiners could see it on the web) but it was interesting and fun to do.

After graduation, the creator of the game and myself still wanted to work on the game to improve it while the other members of the team didn't and left. I tried to improve the Java version, mostly to add proper multiplayer and to rely on websocket to provide real time, two-way communication with the API. However, I've always been a Python guy and choose Java for the first version of the game only because it was more convenient for other team members. I was also a bit tired of the ES5/AngularJS frontend and wanted to switch to ES6 (we were in 2015 at this time, JS already made its "revolution" with ES6). The fact that ES5/AngularJS was what I was using at work also influenced my decision: I wanted to play with something new and different.

So, I decided to rewrite the thing. I tested few options in the backend:

• Go: it felt complicated given what I knew and what I was trying to achieve. Furthermore, I dislike the type system, maybe because I came from a Java and Python background. My main paint point was that I was already using sets in the algorithm and since Go doesn't have sets in the standard library, I needed an extra package. And since Go doesn't support generics, I had to allow any type in my sets which defeats the purpose of a type system in my opinion. That's the equivalent of doing Set<Object> in Java which is strongly discouraged.
• NodeJS: it felt very young and was lacking some modern APIs (like Set) at the time. That would have required to either not use them or to somehow transpile the code which I wasn't willing to do. And the future of the project wasn't very clear at the time with the io.js fork.
• Python: I was able to do all I wanted and it's a language I always used and loved. I was also using it at work, so it made sense to use it here too. Did I really need to test anything else you may ask? For the sake of testing and experimenting I may answer. I choose (and still use today) the Autobahn library for websocket communication. What I found interesting was that I could use it with asyncio (which landed in the standard library in Python 3.4, the Python version of the time) which seemed like the way to go for async programing. That was a very goo choice, mostly in Python 3.5 when the async and await keywords were introduced and completely made asyncio the way to go for new async code.

In the frontend, I tried several options (VueJS wasn't really a thing at this time, so I didn't tested it):

• Angular 2 (now just named Angular): it seemed like the logical choice since the existing app was written in AngularJS. The framework was in Beta at the time. I really didn't like the experience it provided: it felt very complicated and cluttered. I also encounter several problems with the router which didn't invite me to trust the framework. Even after reading a book about it a few months later, after it was officially released, it still felt over-engineered and not that clear to me. So I was glad I discarded it.
• Ember: it felt nice but limiting because of how it handles data. I didn't really adhere to its ways of doing things. Still felt like a strong framework.
• React: it didn't left much of an impression at the time. I don't think it was that popular back then and it felt like I would lack some structure and need to add many things by myself (which is consistent with the fact it's more a rendering library than a framework).
• Aurelia: that's the framework I immediately loved. It felt like an updated and simplified (in a good sense) version of AngularJS. I saw how I could transfer and improve my existing practices of working with AngularJS. I still think it is an under-appreciated framework that deserves to be more known.

So after these tests, I decided to rewrite the project in Python with Autobahn using asyncio and Aurelia in the frontend. I started in July 2015 and I think I completed a first version during the rest of the summer. Of course the project evolved over time:

• I added CI to run my tests automatically.
• I add many linters (both in the backend and in the frontend). I also added prettier and black to automatically format my code. I check formatting with a pre-commit hook in git and even let PyCharm reformat the code on save.
• I switched to pre-commit to manage my pre-commit hooks.
• I switched to Aurelia store to manage the application state. I initially relied on a custom service but relying on a more complete solution made sense in the end. See this article to learn more about this process.
• I added validation in the API. Initially I always assumed I was given proper data (I guess I was young and unafraid back then). By the way, in Python, Cerberus is a very good library to do that!
• I'm stricter in my usage of functions: initially I embraced the flexibility of Python a bit to much and allowed both Card object and tuple (card_name, card_color) (which allowed me to find the proper Card object) to be passed around which made the code more complex and more fragile. I'm stricter now in how I define the interfaces of my functions.
• I reduce various subjects of tech debts and simplified many portions of the code thanks to all the things I learned in Python, JS and programming in general over the last 6 years.
• I changed a few times of package managers and bundles for the frontend. See this article to learn more.

One very important point I haven't talked about yet is the deploy process. That's another area where I learned much over these years! I had a server even as a student, so I used it even for the earliest versions of the game. I don't remember exactly how I deployed the first version of the app using docker though. I think I just exported the image as a tar.gz and imported it on the server with some kind of custom Bash script. Since it was the early days of docker which wasn't very stable at the time, I switched to standard uwsgi directly on the server and packaging all Python dependencies as RPM packages installed on the system (luckily I didn't have much of those because it takes a lot of time!). Later, when I wanted to switch to a more powerful server, I think I switched back to docker because it made dependency management and deploys easier. At this stage, I was still using custom Bash scripts to build and deploy images. The front was deployed with Rsync. It was a bit weird, not always stable but globally worked. After that, I switched jobs and discovered Ansible as a tool to deploy stuff. Since it way more stable and complete than what I could create by hand, I switched to it a few years back. That's where I am now: a bunch of Ansible playbooks to build and deploy both the frontend and the backend. The backend runs in docker and the frontend is served directly by nginx because it doesn't need anything else. I still have a Redis instance that was deployed once, so it doesn't bother me much.

The last thing I'd like to share is how I handle multiple versions of the game. It's not very useful right now since we don't have many players but it felt like something I had to do correctly from the start because it would be harder to add later on. Since the game can change from one version to another (because we had features), incompatibilities can arise: the frontend can send a request in an older format or the API can require new information the frontend cannot send yet. The result would be a broken game for the player. I could use feature flags to detect which code path to use and have a single instance of the API but I think it would clutter the code more than serving the actual purpose. So I decided that each version of the frontend would talk to a dedicated version of the backend. To do that, I developed a small gateway which reads the version of the API the frontend wants and transfer the traffic to the proper container. It's written in Go because I really wanted to give another try to this language and though this routing gateway would be a very good fit for the language (which I still think it is). When you go on the site, nginx will always serve the latest version of the frontend by default. Once you start a game, the version of the frontend is used in the URL so if you refresh the page, you will get the proper version.

To conclude, I really like the project. It took time to go to where we are now for various reasons: I made mistakes because I was a beginner, I can only work on the project during my spare time, I'm mostly the only developer working on the project and we made some changes in the gameplay that had a huge impact on the structure of the project. Overall, I am pleased and proud of the work accomplished. I learned a lot, was able to try thing and spent countless hours over Python, JS, Bash or Docker problems. This may sound like a loss of time but it was a great way to learn. I still plan to work on this project and don't know where it will be in 6 years from now. If it's still around, I'll try to publish a follow up article!