I wrote a GitHub Action using Kotlin

I decided to take a look at GitHub Actions so for the past week I’ve been watching and reading everything about it. I even wrote a post, an Introduction to GitHub Actions. What I found really interesting is the fact that you can write an action using the language you feel more comfortable with. And so I did!

I wrote a small action that:

  1. collects all the changed files from a PR,
  2. keeps those that have the .kt suffix,
  3. runs ktlint on them and
  4. makes a comment in the PR for every error that ktlint reports

and all that using Kotlin and some bash!

You can find and use it here: ktlint-pr-comments

GitHub Action using Docker

There are three ways to create an action but only the one using Docker allows us to use the language we want.

In all three ways the main two ingredients are:

  • the action’s code
  • the action’s metadata, a file called action.yml which is placed in the root folder of your project and defines how the action will run and what inputs/outputs it has.

In our case there is also a third ingredient, a Dockerfile or a docker image which is used by the action’s runner to create a container and execute the action’s code inside it. All you have to do is to make sure that the action’s executable parts are being copied in the container and that are called upon its start.

The runner makes sure that the working space is being mounted to the container (in the state that it was just before the action is started) along with all the environment variables and the inputs the action needs. You can read more in the documentation.

Ktlint PR comments

Action’s code

The action has three distinct parts.

The first part is responsible for using GitHub’s REST API to collect all of the PR’s changes and then keep those that are in Kotlin files and were added or modified. For that I used kscript and I was able to leverage all the libraries that I was accustomed to, like Retrofit and Moshi. When I was happy with the resulted script I used its --package option to create a standalone binary and copy it in the action’s Docker image.

The second part is a combination of bash commands that execute the ktlint binary by passing to it the results of the first part. Ktlint is being called with the --reporter=json parameter in order to create a JSON report.

The third and final part is again a kscript script that uses the report created before and GitHub’s REST API to make a PR line comment for every ktlint error that is part of the PR’s diff. Again a standalone binary was created and put in the image.

Note:

I like kscript since I can write things fast, easy and with all the libraries that I know but I also like writing test first and that proved to be quite difficult. So what I ended up doing was to act as if I was in a Kotlin project. I created my tests (using all my favorites like junit5, hamkrest and MockWebServer) and from that I created .kt files with the proper functionality. And for having a script I created a .kts file where I defined the external dependencies and included the .kt files:

import kotlin.system.exitProcess
//DEPS com.squareup.moshi:moshi:1.9.3
//DEPS com.squareup.moshi:moshi-kotlin:1.9.3
//DEPS com.squareup.retrofit2:retrofit:2.9.0
//DEPS com.squareup.retrofit2:converter-moshi:2.9.0
//INCLUDE logging.kt
//INCLUDE common.kt
//INCLUDE collectPrChanges.kt
//INCLUDE createGithubEvent.kt
val result = collectPrChanges(args)
if (result != 0) {
exitProcess(result)
}
println("Changes collected")
view raw exec-kscript.kts hosted with ❤ by GitHub
Action’s metadata

From the start what I wanted for this action was to be as autonomous as possible leaving very little responsibilities to the consumer and allowing her to just plug it in and watch it play.

For that the only input the action needs is a token, for allowing the kscript scripts to communicate with the API, which will most likely be the default secrets.GITHUB_TOKEN making the action’s usage as simple as adding the following lines in your workflow:

- uses: le0nidas/ktlint-pr-comments@v1
  with:
    repotoken: ${{ secrets.GITHUB_TOKEN }}
Docker image

A lot of things must happen in order to have the action ready to run. Sdkman, Kotlin and kscript must be installed, the code needs to be retrieved from the repository and both kscript scripts have to be packaged. On top of that ktlint must be downloaded and placed in the proper path.

For all that, and to shave a few seconds from the action, I decided to have an image that has everything ready. So I created a workflow that gets triggered every time there is a push in the main branch, builds and packs everything in an image and pushes the result to Docker Hub.

So now the action simply uses that image to run a container without any other ceremonies.

Note:

Before using Docker Hub I tried to use GitHub Packages but it turns out that public is not that public* since it requires an authentication to retrieve a package.

Summary

That’s it! An action for having ktlint’s report as comments in your PR. A result of trial and error since I wrote it while learning about actions but I hope that someone might find it useful. If you do let me know!

An example of how to use it can be found in the action’s repository where I dogfood it to the project.

* [3 Sep 2020]: looks like things will change

Introduction to GitHub Actions

Github actions is a way to execute one or more tasks every time a certain event occurs. For example, setting reviewers (the task) every time a PR is being opened (the event).

How do I use them?

By creating workflows. A workflow is a .yml file that (a) defines the events that we wish to listen for and (b) describes the jobs that are going to be executed. A job is the task that was mentioned earlier and consists from one or more of steps. Now a step is either a command or an action but more on that in a bit.

Lets see a simple workflow to clarify those definitions:

name: Check PR
on:
pull_request:
types: [opened, synchronize]
jobs:
check-pr-quality:
runs-on: ubuntu-latest
steps:
run: echo "Get code from repository"
run: echo "Setup JDK 1.8"
run: echo "Make gradlew executable"
run: echo "Compile project"
run: echo "Run tests"
run: echo "Run quality tools"
set-reviewer:
runs-on: ubuntu-latest
steps:
run: echo "Check availability"
run: echo "Set first available"
view raw demo-workflow.yml hosted with ❤ by GitHub

This is a workflow named Check PR that runs every time there is a new or updated PR and executes two jobs (here in parallel but can be configured to execute them sequentially):

  1. The first job, named check-pr-quality, has 6 steps and in each one runs the echo command to print a message.
  2. The second job, named set-reviewer, has 2 steps and, as in the first one, it runs the echo command in each step to print a message.

Admittedly, this is not a valuable workflow but it is a valid one! Just copy and paste it in a .yml file inside your repository under the .github/workflows folder (this is where all workflows should be added) and then create a new PR. After pushing it to GitHub go to the Actions tab you will see something similar to this:

What’s happening under the hood is that for each job:

  • a new and clean instance of Ubuntu (configured by runs-on key) is being boot up in a virtual machine
  • some necessary environment variables are being set up, and
  • we get “landed” in the /home/runner/work/our-repo-name/our-repo-name (no need to remember it, you can use the GITHUB_WORKSPACE environment variable) path ready to start executing the defined steps

Steps

As mentioned earlier a step can either be a command or an action.

A command is everything that can be executed inside the OS. Let’s not forget that we are inside a virtual machine with various tools installed. We can run from simple bash commands like the one we saw (echo) to multiple ones that help as install and run extra software. For example:

- run: |
    curl -s "https://get.sdkman.io" | bash
    source "$HOME/.sdkman/bin/sdkman-init.sh" && sdk install kotlin

this snippet installs sdkman and then uses it to install kotlin. So now our Ubuntu instance has Kotlin!

An action is what we’ll use when a couple of commands won’t be enough. Think of an action like a full fledged program that can have inputs, produce outputs and can be programmed to do pretty much whatever we want.

Lets change the first job in the workflow above by replacing all echo commands with actions and commands that will actually do what was previously just described:

name: Check PR
on:
pull_request:
types: [opened, synchronize]
jobs:
check-pr-quality:
runs-on: ubuntu-latest
steps:
name: Get code from repository
uses: actions/checkout@v2
name: Setup JDK 1.8
uses: actions/setup-java@v1
with:
java-version: 1.8
name: Make gradlew executable
run: chmod +x gradlew
name: Compile project
run: ./gradlew assemble
name: Run tests
run: ./gradlew test
name: Run quality tools
uses: le0nidas/ktlint-pr-comments@v1
with:
repotoken: ${{ secrets.GITHUB_TOKEN }}
view raw working-workflow.yml hosted with ❤ by GitHub

As before we have a workflow which executes a job that consists from 6 steps.

Only this time we use an action to download our repository to GITHUB_WORKSPACE, an action to setup our java environment, a couple of commands to compile and test our project now that we have it in our workspace (gradlew is part of our project) and an action that will run ktlint and make comments on our PR if there are any errors.

Summary

Think of it like being in a terminal, running and combining multiple commands to produce a result. A job is that terminal and actions are those commands. Mix and match them to automate whatever takes time in your processes.

For more on GitHub actions you can read its documentation, do some of the official training or start reading the code of all those available actions in the marketplace!

Know your tools: scratch files in IntelliJ IDEA

I’ve used scratch files in IntelliJ IDEA and Android Studio but I think that can be found in all of Jetbrain’s products.

What are they?

Scratch files are files that don’t get tracked by the version control system, can be created at any given time and, most importantly, get bind to the IDE and not the project that is currently open.

How do I create them?

The simplest way is to hit ctrl+alt+shift+insert. If you can’t remember it press shift twice and start writing scratch, you will be presented with the action of creating a new one.

The next step is to choose what kind of file you want to create and this is where it gets interesting since you can choose from a plethora of file types. From plain text, to markdown, Kotlin, JSON, XML, ruby and many many more!

How do I use them?

By choosing the file’s type you choose how the IDE will behave when you are working on it, so if you create a scratch.json and paste some json in it you can format it accordingly. Or if you create a scratch.md you can start writing in markdown and have a preview of your work.

But the most powerful aspect of those files is when you create code related ones. If, for example, you create a scratch.kts file and start writing some Kotlin in it, you will see your code being run on the fly presenting to you its result:

TDDish

You can even work test first if you need to figure out a quick algorithm and have your test run in every change you make!

I usually start with an assertThat function and a failing test and go from there:

failing

Its a simple one but you get the point:

passing

TIL: vararg in Kotlin is never nullable

Today I came across a piece of code that looked like this:

fun printAll(vararg names: String?) {
  names.forEach { name -> println(name) }
}

and noticed that the IDE did not complain about using names without checking if it is null names?.forEach { ... }!

After decompiling Kotlin’s bytecode I saw that no matter what type I use (String? or String) the java code was the same:

public static final void printAll(@NotNull String... languages) {
  //...
}

Does the Kotlin compiler ignore the nullable type completely? Turns out that yes!

And the reason is quite clear and straightforward but it hadn’t registered in my mind until now:

Note that vararg parameters are, as a rule, never nullable, because in Java there is no good way to distinguish between passing null as the entire vararg array versus passing null as a single element of a non-null vararg array.

kotlin’s forums

What if we pass a null value?

As a matter of fact if you pass a null value:

printAll(null)

the compiler makes sure that the java code will be called with a null value casted to the appropriate type:

printAll((String)null);

which ends up in an array of strings that has one element!

kscript from docker

This is one of those cases that you go down the rabbit hole and end up doing ten different things before the one that you initially wanted to!

My original intention was to try out kscript. What I ended up doing is learning about dockerfiles, building images and trying to run .kts files without having kscript, or kotlin, installed locally in my machine!

The result

A github repository that you can clone, run the install script and have a kscript executable that works as described here.

Why?

I wanted to play with kscript but did not want to install it on my machine. I don’t like bloating my OS by having libraries and software that might not be used often.

How?

Having everything installed in a docker image and use containers to play with kscript was a one way street for what I wanted to achieve.

Dockerfile

It was my first dockerfile but there are tons of tutorials that helped me out. I ended up using alpine, installing java through its repositories, sdkman using its installation guide and kotlin, kscript through sdkman!

Finally, kscript was added as the image’s entry point which means that every time a container is being run kscript will be the first command that gets executed.

And it worked for the most parts. By executing

docker run –rm -i kscript <params>

rm to remove the container after its done, -i to have the script’s output passed to the terminal

I was able to use kscript as described in its repository, except for the case of having a .kts file!

kscript-router.sh

For the case of a .kts file what turned out to solve my problem was to pass the file’s entire content. And this is part of what kscript-router.sh does. It checks if the first parameter is a .kts file and if so it passes its contents to the container. In every other case it passes all given parameters directly to the container.

install.sh

To tie everything together I added a script that creates the image, adds the router script to the user’s path and creates an alias, named kscript, for executing the router.

kscript-from-docker

Readable codebase: extract conditions / predicates

A quick way to make a codebase more readable is to extract long or complex conditions / predicates to their own functions. The function’s name will explain to the reader what is being checked allowing her to move forward instead of trying to figure out the context or the calculations that are taking place.

The task

enum class Status {
NotStarted,
InProgress,
Resolved,
Cancelled
}
class Task(
val id: Int,
val description: String,
val status: Status,
val createdAt: LocalDateTime
)

Completed tasks of the last two days

Lets assume that we have a piece of code that prints all tasks that were created in the last two days and are, as the business experts calls them, completed. Which means that are either Resolved or Cancelled.

val yesterdayAtStartOfDay = LocalDate.now().minusDays(1).atStartOfDay()
val todayAtEndOfDay = LocalDate.now().atTime(23, 59)
allTasks
.filter { task >
(task.createdAt > yesterdayAtStartOfDay && task.createdAt < todayAtEndOfDay)
&& (task.status == Resolved || task.status == Cancelled)
}
.forEach { task > println(task.description) }

We have two distinct concepts that can be extracted. One that will explain the date calculations and one that will force the business knowledge behind those two statuses.

createdInTheLastTwoDays()

The calculations can be extracted to a private function but since we are using Kotlin we will extract them to an extension function to make the code even more readable:

allTasks
.filter { task >
task.createdInTheLastTwoDays()
&& (task.status == Resolved || task.status == Cancelled)
}
.forEach { task > println(task.description) }
// extracted code:
private fun Task.createdInTheLastTwoDays(): Boolean {
val yesterdayAtStartOfDay = LocalDate.now().minusDays(1).atStartOfDay()
val todayAtEndOfDay = LocalDate.now().atTime(23, 59)
return this.createdAt > yesterdayAtStartOfDay && this.createdAt < todayAtEndOfDay
}

isCompleted()

Another benefit of having “the mentality of extraction” (yep, I just made it up) is that eventually we will come across a case like the completed which is a business term that has not been transferred to the codebase!

So we don’t just make the code more readable but we improve our domain representation and force a rule that until now was known through documentations or, even worse, through conversations only.

allTasks
.filter { task > task.createdInTheLastTwoDays() && task.isCompleted() }
.forEach { task > println(task.description) }
//extracted code:
class Task(
val id: Int,
val description: String,
val status: Status,
val createdAt: LocalDateTime
) {
fun isCompleted(): Boolean {
return status == Resolved
|| status == Cancelled
}
}

The reader will not have to make any calculations while reading this code or trying to understand why only those statuses are being checked.

The code speaks for itself!

Write comments only when they answer the “Why?”

This is the one and only question that I ask myself when I see comments or when I feel that I need to write some:

Do these comments provide the reason why this code was written?

If not then 99.9% we don’t need them.

“Remove right margin”

I recently came across a block of code that looked like this:

private fun foo(ui: Ui, system: System) {
val isGridLayout: Boolean = ui.itemsAreInGrid()
// Remove right margin when the theme is light and the layout is a list
if (!isGridLayout && !system.isDarkThemeEnabled()) {
val uiContainer = ui.container
uiContainer.rightMargin = 0
}
}

In this example the comment simply describes what the code does. I am pretty sure that everyone can figure that out but only the code’s author knows why we need to remove the margin under those circumstances. I am also sure that in six months (in my case in six days) even the author won’t remember the reason behind this code.

Extract if’s logic

To be fair I do understand the author’s urge to write this comment. Those unavoidable negations, because of the system’s/ui’s APIs, look like they need to be clarified.

Fortunately there is a better way to do that: we extract if’s logic to its own method and name it accordingly:

private fun foo(ui: Ui, system: System) {
// Remove right margin when the theme is light and the layout is a list
if (inLightModeWithItemsInList(ui, system)) {
val uiContainer = ui.container
uiContainer.rightMargin = 0
}
}
private fun inLightModeWithItemsInList(ui: Ui, system: System): Boolean {
return !ui.itemsAreInGrid() && !system.isDarkThemeEnabled()
}

Extract if’s body

We can go a step further and extract if’s body to its own method too, having the code replacing the comment completely:

private fun foo(ui: Ui, system: System) {
if (inLightModeWithItemsInList(ui, system)) {
removeRightMargin(ui)
}
}
private fun removeRightMargin(ui: Ui) {
val uiContainer = ui.container
uiContainer.rightMargin = 0
}
private fun inLightModeWithItemsInList(ui: Ui, system: System): Boolean {
return !ui.itemsAreInGrid() && !system.isDarkThemeEnabled()
}

Answer the “Why?”

So now what is missing is the reason behind the margin’s removal. A comment that answers the why and completes the reader’s understanding about the code she/he reads.

private fun foo(ui: Ui, system: System) {
if (inLightModeWithItemsInList(ui, system)) {
// blah blah …
removeRightMargin(ui)
}
}

“I can see what is happening and I know why too!”

Code review: don’t just request a change

When reviewing a PR I try to elaborate on my proposals for two reasons:

1. Out of respect to my colleague

Every piece of code is part of an effort that took time and thought. Requesting a code change by simply asking it (i don't like it, change it) or by dictating it (do it this way) demotes all the work that has been done.

I figured that, since my colleague spent a few hours to come up with a solution I owe her/him more than a few seconds!

2. I solidify my knowledge or even better, learn something new

By trying to justify the reason behind a request I end up understanding better why I prefer a solution or a format over another.

When I provide an example or when I do a mini investigation to help me clarify why something must change I often find that (a) many things that I thought I knew were not as I had them in mind and (b) I now understand a concept much better because I had to write about it. You don’t master something if you can’t explain it!

PS: never forget that it might say “request a change” but in reality you start a conversation between two professionals that will eventually benefit of the project

Cherry pick all the commits from a merged branch

I recently had to revert an entire feature and create a new branch that would be merged in the future. In git terms, I had to revert my merged branch, create a new one and cherry pick the reverted commits to it.

I’ve been working with git for years but up until now I never had to revert a commit, let alone an entire branch. Turns out reverting is quite simple and can be done in a single command but cherry-picking all the branch’s commits is a repeatable and tedious task since we need to copy each commit, create a string out of them and give it to cherry-pick.

What I wanted was a way to give git the merge commit and let it figure out which commits to cherry-pick. For example, in the case described below:

I would like to cherry-pick all five commits edaa436, 5d48f23, a8f2b4d, 63f2f4e, 1dc3f6a by just providing to git the commit f51c08c.

I had already done a “custom” git-alias so I figured I could do something similar and create cherry-grab: git cherry-grab f51c08c

commit^

Two things helped in creating cherry-grab and the first one is the ^ suffix that one of its usages is to get the commit’s parent. So if we run git show edaa436^ git will translate it to git show 84f8f45.

In our case where the commit is the result of a merge we can specify which of the two parents we want. f51c08c^1 will be translated to 84f8f45 and f51c08c^2 to 1dc3f6a (in a similar way if we need the nth parent we request it by using ^n).

git rev-list <commit>

The second thing was rev-list which returns a list with all the commits that can be reached through the parent “pointer” starting from the provided commit. So if we run git rev-list edaa436 git will return edaa436, 84f8f45 and 55da68d.

Also, rev-list can exclude all commits that are reachable from a given commit as long as we prefix it with the caret (^) symbol. So if we run git rev-list edaa436 ^84f8f45 git will return just edaa436.

cherry-grab

First we need to get the list with all of the branch’s commits. To do that we will use the rev-list command and request all commits starting from the merge-commit’s second parent but exclude all commits that are reachable from the merge-commit’s first parent:

git rev-list 1dc3f6a ^84f8f45

this returns 1dc3f6a, 63f2f4e, a8f2b4d, 5d48f23, edaa436 which are the commits that construct the merged branch.

Then we need to reverse this list so that we can apply each commit in the proper order:

git rev-list 1dc3f6a ^84f8f45 | tac

which returns edaa436, 5d48f23, a8f2b4d, 63f2f4e, 1dc3f6a.

Having the commits in the correct order we just need to pass them to the cherry-pick command to apply their changes:

git rev-list 1dc3f6a ^84f8f45 | tac | xargs git cherry-pick

Finally we need to put in an alias to make it reusable:

[alias]
cherry-grab = "!f() { \
git rev-list $1^2 ^$1^1 | tac | xargs git cherry-pick; \
}; f"

Nothing fancy or special, we just wrap the sequence of command with a function and tell git to call that function when ever we use the cherry-grab alias.

In order to make it reusable we replace the parent commits with $1^2 and $1^1 respectively which are translated to the second and first parent of the passed commit.

That’s it:

Git alias with parameters

In the company I work we use the gitflow workflow and in every feature branch we make sure that part of its name is the task’s id which includes a unique number. So what we end up with is something like:

feature/T12345-make-the-app-rock

What I wanted was an easy way to checkout a branch using just the number. I wanted (spoiler: and succeeded) something like this:

git ch 12345

Terminal

Lets see how we can checkout a branch without an alias but by using the task’s number.

Get the branch

We can get the branch by listing all branches (git branch) and then grep for the one that contains the task’s number:

git branch | grep 12345

this will return feature/T12345-make-the-app-rock

Checkout to it

Having a way to get the branch makes the checkout quite easy since we can use bash’s command substitution to execute the above commands and use the result directly in the checkout:

git checkout $(git branch | grep 12345)

Put it in a function

We are now able to checkout the task’s branch but it would be nice if we could change the task’s number more easily. Lets create a function and pass the number as a parameter:

f() {
git checkout $(git branch | grep $1)
}
view raw f.bash hosted with ❤ by GitHub
  • The $1 is where the parameter will be used (if we had more parameters we would use $2 for the second, $3 for the third etc).
  • You can check the function by writing it directly in the terminal. Just make sure that you add newlines (press enter):

The alias

We can change branches easily but once we close the current terminal session we lose everything. Our goal is to have all that in a git alias.

Creating an alias can be achieved either by using git config

git config –global alias.ch checkout

or by doing it manually and adding the alias directly into the .gitconfig file (usually located in the home directory). The file should end up with something like:

[alias]
ch = checkout
view raw checkout.gitconfig hosted with ❤ by GitHub

Having just the alias will not help us much. As is, it will work only if we give it the entire branch:

git ch feature/T12345-make-the-app-rock

What we need is a way to connect the function that we created with the alias.

The answer is provided by git itself since it has a way to create aliases that do not execute a git command but an external command:

However, maybe you want to run an external command, rather than a Git subcommand. In that case, you start the command with a ! character. This is useful if you write your own tools that work with a Git repository.

git docs

Following the docs we can change the gitconfig file to look like this (this time the change must be made only manually because our function will need to have newlines):

[alias]
ch = "!f() { \
git checkout $(git branch | grep $1); \
}; f"

What will happen when we write git ch 12345 is:

  1. git ch will be replaced with everything inside the "..." (minus the !) ending up with two commands
  2. the first one creates the f function and
  3. the second one calls f with the parameter 12345: f 12345

And that’s it! We now have a git alias that will allow us to checkout a branch using part of its name.

feature image by Zach Reiner @ unsplash