This is an updated version of the 2019 post, architecture of my life, which you can find here.
Introduction
Automation is something dear to my heart. Like the figure at the top of this post, I am not sure that in the end it will have actually saved me time, but it has given me many opportunities to learn, as trying to build a resilient piece of software with a beautiful architecture, which helps me be more productive is an ever-evolving project, which will keep pushing me to learn more. I hope with this I can inspire you to automate some part of your life.
In this blogpost I will describe the following things:
- The foundation: The code my automation is built upon. It will also explain the fundamental ideas behind how I think about automation.
- Actions: An overview of some of the automation I have built. If you're looking for some inspiration to build something yourself, this is the place to go.
- Other software: Not everything I built can be run inside the framework I built. I also built some other cool utilities to help me in my every day life.
The foundation
The core and foundation of my automation is named Atlas. Atlas handles everything to run my automation, from connecting to services to handling errors.
An Action
At the core of Atlas is a simple idea: Actions. Anything I might ever want to automate is an action. It will act on my behalf on the online services that I use. Some actions will check for themselves if they need to run (adding a reminder for a sent email). The action can also decide how often it wants to run, from every 15 minutes to once a week. Others I only want to run when I need them to (checking the uptime of my systems).
Some require input to do their job correctly (adding a todo to my todo list). The most complicated actions have some state, as the input they require is more complicated.
To build this state I use a chatbot interface, where I give a command, the system parses my input, updates the state and performs the next step, incrementally building the state until all the necessary properties for the action to complete are in place. This flow is explained in the figure below
For example, when I want to add a task to a specific task list, I first write the content of that task, which is added to the state. It replies asking which project I want it to add to with some example task lists. I reply with "Write a blog". In the "Update State" part of the flow, this title of a task list is then internally converted to the internal id of the list. This way, me and anyone else using the system do not need to know the exact internals.
Interface
Atlas is always part of another project. It is wrapped in Ares, a project that connects Atlas and the Microsoft Bot Framework to make it available as a chatbot, in my case as a Telegram chatbot, where you can ask it to perform the actions. It is also run as an Azure Function, available as a web-api. I have used this API to build an iOS shortcut to run Actions from my phone and watch and have created a windows application such that I can run any action from my laptop. The Azure Function also runs on a timer trigger and executes every 15 minutes to execute the relevant actions.
Exceptions
One of the problems of building your own automation is that things are breaking. Constantly. I am connecting to 17 online services and having 35.000+ lines of code means that it is likely something will break. Most of the errors occur within Actions, as they make the calls to external services and change (and therefore break) most often. To make sure one action does not break the entire system, the actions are run in a try-catch block.
Once an action crashes, automation will start running to handle the crash. Independently running actions will start to send me messages to tell me that it is broken after four failed runs, so that timeouts and other temporary problems don’t bother me. It will also automatically throttle itself, postponing its next run in an exponential way, to make sure it does not break anything and I do not get spammed with messages.
After 10 crashes, my automation will automatically create a GitHub issue with the name of the action that crashes and the stack trace. It will even try to generate a url to line in the code in GitHub link where it believes the issue is originating.
The Telegram chatbot will ask the user after a single crash if it wants to make a GitHub issue, as this is deemed more important.
Now, you have a good overview of the basics, we can discuss the cool parts.
Actions
Spotify
I like to listen to a lot of music. Last year, I spent 98.960 minutes listening to Spotify, one of the most popular streaming services out there. To help me get the most out of their service, I have a lot of automation running to help me.
Sort by Loudness
To aid you in finding new music, Spotify presents you with two auto-generated personalized playlists. A Discover Weekly playlist with new music from new artists and a Release Radar playlist which contains music released this week. Both are very nice, but have one problem, they jump from very loud energetic to peaceful piano music. This is very jarring if you are listening. To solve this, I download all songs from the playlist and get the features of the songs. These features are provided by Spotify and contain things like energy, danceability, liveness and loudness. I order all songs by loudness and put them in a new playlist for me. This way I can listen to my music from very loud to very peaceful.
Personally generated playlists
Once you have features of songs, you can think bigger. Spotify can extract features from all of my saved music. From that you can generate playlists. However to do that, you would need to find the perfect settings to create a nice playlist. For this I created a windows application called Playlister, explained further below. Using Playlister, I have created playlists named Summer, Winter, Piano and Energy. Every week, Atlas downloads all my saved songs from Spotify and runs it through each of the finely tuned filters for my playlists and updates each with new songs I’ve added and removes songs I no longer like. So if I want to listen to music that gets me hyped, I listen to the Energy playlist, which consists of songs I like that, following my tweaks, can be considered full of energy.
Last 4 weeks
I (used to) spend a lot of time biking and travelling by train. I have a data plan, but don't want to overshoot it. This leads to the fact that I need Spotify to download a lot of music. However, I also have a lot of new music that I want to bring with me. I could download all my music, but my phone does not have enough storage for that. To solve this problem, Atlas will update a playlist every night with all of the songs I added in the last 4 weeks. I then tell Spotify to keep this playlist downloaded at all times. This leads to an almost always updated playlist with my newest songs.
Random Songs
In the previous paragraph I explained that I can’t download every songs to my phone. However, I might want to listen to a random selection of music every once in a while. Music not in one of my automatically generated playlist or in any other list. Furthermore, even if I have internet, I sometimes feel like the shuffle is not completely random. And lastly, it is also sorted by loudness and if I need to concentrate, I can start at loud and end at calm music. To solve these problems, every night Atlas takes all my saved music, picks 50 random songs and adds it to a playlist sorted by loudness, so I can always listen to some random music I like.
Full Release Radar
Spotify has a feature called Release Radar, where each week, you can listen to music that came out that week from artists you follow or might find interesting. This is great and I’ve gotten a lot of new music from it, however it might miss artists that I like and if an artist released an entire album, it will only display one song in the playlist, which I can definitely understand, but I don’t want. So I created an Action on Atlas, which every Friday checks all the artists I follow. If they released any new music in the last week and it puts the songs into a playlist called “Full release radar”, so that I do have this overview. As an addition, another action checks if I added music from a new artist and automatically follows them, so this list is constantly expanding.
Todoist
Task lists are must for me. I forget a lot of things and task lists enable me to keep track of all the things I need to do. To help me I use Todoist.
Temporary Projects
As some might know, every Friday, I try to clean my room. This is very nice, as during the week I don’t have to think about keeping everything clean, as on Friday I will clean it anyways, nor do I have to force it in sometime during the week. Cleaning is something that I have to go through often and has a predetermined list of tasks. Although I try to do it on Friday, it might occur that it happens on another day. To be able to start with all of these tasks on the fly I have an Action which can import a template at any time. The template is written in TaskPaper, a simplistic way of writing down tasks, to create all projects and tasks. These configuration files are stored in my OneDrive, so I can edit, delete or add any new list at any time or any place.
GitHub Synchronisation
On GitHub, I currently have 7 repositories where I have issues open that I plan on fixing one day. To keep track of this in one place, I synchronize them to Todoist, where I can see them in a separate project. This is only one way, as I neither want to create issues in Todoist, not be able to complete them.
Today action
When doing work from my task list, there are two main categories this work can be divided into: work that needs to be done today and work that you plan on doing today. The first is easily managed in Todoist by due dates. The second is managed by myself with a label @today. Any task with that label and a due date of today is then shown to me and I will work on that on the day. To help me manage this planned work, I have some automation running. Every night an Action runs, which:
- Moves all tasks with the label @today to @yesterday
- Moves all tasks with the label @tomorrow to @today
- Updates all tasks with due date today and the label @planned to remove that due date and adds label @yesterday
This makes sure that when I wake up, I know exactly what I need to do on a day.
Microsoft (Email, Calendar & Teams)
All of my friends know that I send a calendar invite for every dinner/meeting/meetup I have. I have found that this enables everyone to have the same basic truth and then there can be no doubt when a meeting event is. So I make a lot of them and use automation to make it easier.
Event Location
In Outlook, you can also store where a person lives. When I have a meeting at a friends place and their address is in my system, I only have to write the name of the person in the location field of the event and Atlas will add the exact address of that person. During these lockdown times, meetings are now online. If I put “@{Teams}” as the location in the calendar event, it will create a new teams meeting for me and add the link in the calendar event.
Birthday Reminder
Just like every other person on this planet, I am terrible at remembering birthdays. To make sure I don’t forget to buy presents, I get a reminder a week ahead of a persons birthday and then on the day itself to make sure to wish them a happy birthday.
Archive Read
I use inbox zero which results me moving email which I completed or are no longer relevant to an archive folder. To remove this small OCD thing that there might be unread email in that folder, it just marks everything in that folder as read.
GOT invitations
Since the second year I have been studying in Delft, every Thursday me and a group of friends have met up to eat together. Since that time, every week I have sent an invite with the date and who is supposed to be hosting that week. As making that invite every week is a bit annoying and I sometimes very awkwardly would forget someone or use the wrong email address, I made this an Action I can trigger to fill in the location and title and it will handle the rest.
Lights
Recently, I got some smart lights as a present. I now have 5 dimmable lights and 1 light which can change color. Obviously I automate this.
Moods
Changing the lights in your room based on your needs is very nice. To do this I implemented the concept of moods. During the day all lights are at full strength and illuminate my room as possible as I like a bright environment to work. When I enter a meeting, I have a mood which tries to maximize the lighting so I look as good as possible on a video call. When it becomes night, it dims the lights, so I get a bit sleepy. Finally it has an off mood, for when I'm not there or asleep.
Party mode
From the section about Spotify, we learned you can extract features from songs. What’s more is that you can also extract the beats of a given song. This means that you can get the tempo of a song. With Spotify, we can also see which song is playing and where we are in the song. Next, by contacting the smart bulbs, we can toggle the lights on the beat of a song. The lights change light level, one by one, so it looks like it is going from right to left on the beat of the music. It is amazing to experience and very fun to show off to visitors of your room.
Notion
Notion is starting to become my knowledge base for everything. It contains my blogposts, recipes, notes and serves as my Zettelkasten (at least I try to). To help me keep things organized, I have some automation running. As connecting to Notion has proven difficult I have reversed engineered their code to be able to interact with this service.
Recipes
In Notion, I have a list of all the recipes I want. As having them in one single website, disables you from having recipes from multiple sources, having them in Notion is the best other option. Using my automation, I am able to import a new recipe in a few seconds.
Links
I am trying to create a useful knowledge base of all the things I consume. As I'm getting older, I am realizing that things I read and listened to years ago seem new to me again and I want to retain some of the knowledge and wisdom I learned from articles and podcasts, so that I am able to use them at a later point. Therefore I have started collecting them in Notion and am using some automation to template some things away for me.
Other software I made
Startpage
As you might have read in the Todoist section, I have a list of tasks that I want to complete every day. To keep me focused on these tasks, I have created a startpage for myself. It is very minimalistic and features automatic dark mode. When I open a new page, I can also complete a task. Once all tasks are completed, it loads a random image from Unsplash as background.
Daily Todo
To help decide which tasks I want to do today, I created an universal windows app called DailyTodo. I can search for any task, filter by project or label. Once I have picked the task, it will add the @today label to it. This tells my system that I want to do this task today. It will then also make a notification in my windows action center and forces it to stay at the top. So if at any point of the day I wonder what I should do, I can simply open it and quickly pick my next task. As you can also see in the images below, with these notifications I can also quickly complete or remove them from my day.
lnk.nntn.nl
Remembering links can be difficult. When creating a Microsoft teams link, it might not even come across as a valid link for some software. To help you can use a link shortener. There exist a lot of link shorteners already, but I like to be in charge of any personal links that I have. Therefore I created lnk.nntn.nl. As an example, take a look at my CV.
Events website
As written in the Microsoft section, I have a lot of calendar invites. However, sometimes you might want to create an event, where you are not sure yet who wants to show up. To help me fix that problem, I created a website where I can have people sign up for any event that is in my calendar. You write down your email and it will send you the calendar invite. If this is your first time using it, it will ask me to verify your email, so that you cannot spam random people.
This blog
Yes, you read this right. This blog is part of my automation. Using Notion, I am able to write any blog post that I want. Using Atlas, I extract the text from Notion and convert it to html to display on the blog you are reading right now!
Wallsetter
One of the more cute examples of my own code is Wallsetter. This is a small app which I built to set a new lock screen and wallpaper every two hours. It gets new images from Unsplash, an amazing website to find free to use images. Wallsetter is then able to apply all sorts of effects to it.
The cool thing is that for the lock screen is that it is able to check my Trakt and look at which TV show airs today or if there are any shows from yesterday that I have not watched yet. Then it will grab an image of that show and set it as my lock screen or wallpaper.
However, that is not my setup currently I use it to download an image of a cute dog, set it as my wallpaper and set a black and white version of that as my lock screen image. It is gorgeous and I am very happy with it.
Playlister
Spotify can extract features from all of my saved music. From that you can generate playlists, as we have seen before. However to do that, you would need to find the perfect settings to create a nice playlist. To do this, I created Playlister, a universal Windows app, which enables me to play with all the features you see on the right of the image below.
