Showing posts with label Ktor. Show all posts
Showing posts with label Ktor. Show all posts

Sunday 16 October 2022

Kotlin / Ktor: G'day world from a Docker container

G'day:

Not sure what this article is going to end up being about. However I am hovering over the "New Project" button in IntelliJ, and am gonna attempt to at least get to a "G'day world" sort of situation with a Ktor-driven web service today.

Why Ktor

We need to port our monolithic CFML/CFWheels app to a more… erm… forward-thinking and well-designed solution. The existing app got us to where we are, and pays our salaries, but its design reflects a very "CFML-dev" approach to application design. We've decided to shift to Kotlin, as you know. We also need to adopt some sort of framework to implement the application atop-of, and we've chosen Ktor for a few reasons:

  • It's focus is micro-services and small footprint.
  • From what I've read, it focuses on being a framework instead of being an opinion-mill, how other frameworks can tend to be.
  • It's written for Kotlin; unlike say Spring, which is written for Java and it shows. We're using Kotlin to have the benefits of the JVM, but to steer clear the Java Way™ of doing things.
  • It's created by JetBrains, who created Kotlin, so hopefully the Ktor design team with be aligned with the Kotlin design team, so it should be a pretty Kotlin-idiomatic way of doing things.
  • Support for it is baked-in to IntelliJ, so it's a "first class citizen" in the IDE.

Also basically we need to pick something, so we're cracking on with it. If we do some quick investigation and it turns our Ktor ain't for us: I'd rather know sooner rather than later.

Let's get on with it.


Project

One can create a new Ktor project via IntelliJ ("New Project"):

I've only filled in the situation-specific stuff here, and left everything else as default. I've clicked the "Create Git repository" option: I hope if gives me the option to provide a name for it before it charges off and does it, cos I don't want it just to be called "gdayworld". So I might back out of that choice if it doesn't work for me:

Let's press "Next"…

Argh! I have to make decisions! I haven't even finished my first coffee of the day yet!

There are roughly one million plug-ins on offer here, and I don't even know what most of them are. For now, all I need this thing to do is to have testing for a greeting endpoint that says "G'day world" or something, so I doubt I'll need most of this stuff. Let's have a scan through.

OK, I've selected these ones:

  • Routing
  • DefaultHeaders
  • CallLogging
  • CallId
  • kotlinx.serialization - this also required the ContentNegotiation plug-in

After clicking "create" it got on with it, downloaded some stuff, built the project and declared everything was fine. I now have this lot:


Baseline checks

Right, let's see what tests it installed by default:

package me.adamcameron

import io.ktor.server.routing.*
import io.ktor.http.*
import io.ktor.server.plugins.callloging.*
import org.slf4j.event.*
import io.ktor.server.request.*
import io.ktor.server.plugins.callid.*
import io.ktor.serialization.kotlinx.json.*
import io.ktor.server.plugins.contentnegotiation.*
import io.ktor.server.application.*
import io.ktor.server.response.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
import kotlin.test.*
import io.ktor.server.testing.*
import me.adamcameron.plugins.*

class ApplicationTest {
    @Test
    fun testRoot() = testApplication {
        application {
            configureRouting()
        }
        client.get("/").apply {
            assertEquals(HttpStatusCode.OK, status)
            assertEquals("Hello World!", bodyAsText())
        }
    }
}

Most of those imports aren't necessary btw, that's a wee bit sloppy. It only claims to need these ones:

import io.ktor.http.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
import kotlin.test.*
import io.ktor.server.testing.*
import me.adamcameron.plugins.*

I'll leave it as-is for now. The test looks sound actually. Well: I've purposely not looked at the code yet, but a test that tests that a GET to / returns "Hello World!" seems reasonable. Let's run it:

Cool. OK, let's run the app then, given it looks like it'll work:

C:\Users\camer\.jdks\semeru-11.0.17\bin\java.exe […]
2022-10-16 11:01:49.119 [main]  INFO  ktor.application - Autoreload is disabled because the development mode is off.
2022-10-16 11:01:49.205 [main]  INFO  ktor.application - Application started in 0.148 seconds.
2022-10-16 11:01:49.205 [main]  INFO  ktor.application - Application started: io.ktor.server.application.Application@57312fcd
2022-10-16 11:01:50.448 [DefaultDispatcher-worker-1]  INFO  ktor.application - Responding at http://127.0.0.1:8080  

It ran. Does it actually respond on http://127.0.0.1:8080?

Cool. OK, so I have a test that passes an an app that works. Gonna push that to GitHub as v0.2 (v0.1 was the empty repo). And I'm gonna have a shufti around the files it's created and see what's what.


Tweaking

OK, I'm not gonna look at those old-school xUnit-style tests. I'm gonna adapt them to use the more declarative BDD style I've been using so far when testing stuff with Kotlin. So this means I'm going to add some Kotest dependencies. The test is now:

@DisplayName("Tests of the / route")
class ApplicationTest {
    @Test
    fun `Tests the root route responds with the correct status and message`() = testApplication {
        application {
            configureRouting()
        }
        client.get("/").apply {
            status shouldBe HttpStatusCode.OK
            bodyAsText() shouldBe "Hello World!"
        }
    }
}

I'm also refactoring the class name and location to src/test/kotlin/acceptance/IndexRouteTest.kt. It's not testring the app, it's testing the route. Plus it's an acceptance test, and I wanna keep those separate from unit tests / integration tests etc (poss premature optimisation here I guess). I've also lost the subdirectory structure from /src/main/kotlin/me/adamcameron/Application.kt to be just /src/main/kotlin/Application.kt. Kotlin's own style guide recommends this:

In pure Kotlin projects, the recommended directory structure follows the package structure with the common root package omitted. For example, if all the code in the project is in the org.example.kotlin package and its subpackages, files with the org.example.kotlin package should be placed directly under the source root, and files in org.example.kotlin.network.socket should be in the network/socket subdirectory of the source root.

Next I feel there's a design bug in the index route, but I'm gonna push my current tweaks first, and sort that out in the next section.


Giving control to a controller

This design bug: here's the entirety of the implementation of that index route and its handling:

fun Application.configureRouting() {

    routing {
        get("/") {
            call.respondText("Hello World!")
        }
    }
}

That's in /src/main/kotlin/plugins/Routing.kt

Routing should limit itself to what it says ion the tin: routing. It should not be providing the response. It should route the request to a controller which should control how the response is handled. I know this is only example code, but example could should still follow appropriate design practices. So erm: now I have to work out how to create a controller in Ktor. I'm pleased I have a green test on that index route though, cos this is all pretty much a refactoring exercise, so whatever I do: in the end I'll know I have done good if the test still passes.

Hrm. Having not found any examples in the Ktor docs of how to extract controller code out of the routing class, I found myself reading Application structure, specifically these paras:

Different to many other server-side frameworks, it doesn't force us into a specific pattern such as having to place all cohesive routes in a single class name CustomerController for instance. While it is certainly possible, it's not required.
Frameworks such as ASP.NET MVC or Ruby on Rails, have the concept of structuring applications using three folders - Model, View, and Controllers (Routes).

My emphasis. I see. Ktor does not separate-out the idea of routing from the idea of controllers, I see. To me they're different things, but I guess I can see there's overlap. I'm not hugely enamoured with their thinking that "despite the rest of the world using the term MVC, we know better: we're gonna think of it as MVR". Just… why. If you wanna conflate routing and controllers, yeah fine. But in that case they conflate into the controller part of MVC. You don't just go "ah nah it's MVR, trust me". Remember what I said before about opinionated frameworks? This is why I don't like it when frameworks have opinions.

But anyway.

We can still separate out groups of "route-handlers" (sigh) into separate functions. ow I have this:

package routes

import io.ktor.server.application.*
import io.ktor.server.response.*
import io.ktor.server.routing.*

fun Route.indexRouting() {
    route("/") {
        get {
            call.respondText("Hello World!")
        }
    }
}

And my original configureRouting function is just this:

fun Application.configureRouting() {

    routing {
        indexRouting()
    }
}

That's good enough.


Auto-reload

One good thing my RTFMing about controllers lead me to was how to get my app to rebuild / reload when I make code changes. By default every time I changed my code I had to shut down the app (remember it's serving a web app now), rebuild, and then re-run the app. That was not the end of the world, but was pretty manual.

Ktor have thought about this, and the solution is easy.

First, I tell my app it's in development mode (in gradle.properties):

junitJupiterVersion=5.9.0
kotestVersion=5.5.0
kotlinVersion=1.7.20
ktorVersion=2.1.2
logbackVersion=1.2.11

kotlin.code.style=official

org.gradle.warning.mode=all

development=true

This in turn is picked up by code in build.gradle.kts

application {
    mainClass.set("ApplicationKt")

    val isDevelopment: Boolean = project.ext.has("development")
    applicationDefaultJvmArgs = listOf("-Dio.ktor.development=$isDevelopment")
}

(that code was already there).

Then I needed to tell the app what to pay attention to for reloading (in Application.kt):

fun main() {
    embeddedServer(Netty, port = 8080, host = "0.0.0.0", watchPaths = listOf("classes")) {
        configureMonitoring()
        configureSerialization()
        configureRouting()
    }.start(wait = true)
}

classes there is a reference to build/classes in the project file system.

Then get Gradle to rebuild when any source code changes:

PS C:\src\kotlin\ktor\gdayworld> ./gradlew --continuous :build
BUILD SUCCESSFUL in 1s
13 actionable tasks: 13 up-to-date

Waiting for changes to input files... (ctrl-d then enter to exit)
<-------------> 0% WAITING
> IDLE

And another instance of Gradle to run the app with it watching the build results:

PS C:\src\kotlin\ktor\gdayworld> ./gradlew :run                  
> Task :run
2022-10-16 14:21:19.374 [main]  DEBUG ktor.application - Java Home: C:\apps\openjdk\EclipseAdoptium
2022-10-16 14:21:19.374 [main]  DEBUG ktor.application - Class Loader: jdk.internal.loader.ClassLoaders$AppClassLoader@73d16e93:...]
2022-10-16 14:21:19.390 [main]  DEBUG ktor.application - Watching C:\src\kotlin\ktor\gdayworld\build\classes\kotlin\main\me\adamcameron\plugins for changes.
2022-10-16 14:21:19.390 [main]  DEBUG ktor.application - Watching C:\src\kotlin\ktor\gdayworld\build\classes\kotlin\main\routes for changes.
2022-10-16 14:21:19.390 [main]  DEBUG ktor.application - Watching C:\src\kotlin\ktor\gdayworld\build\classes\kotlin\main\me for changes.
2022-10-16 14:21:19.390 [main]  DEBUG ktor.application - Watching C:\src\kotlin\ktor\gdayworld\build\classes\kotlin\main for changes.
2022-10-16 14:21:19.390 [main]  DEBUG ktor.application - Watching C:\src\kotlin\ktor\gdayworld\build\classes\kotlin\main\META-INF for changes.
2022-10-16 14:21:19.390 [main]  DEBUG ktor.application - Watching C:\src\kotlin\ktor\gdayworld\build\classes\kotlin\main\me\adamcameron for changes.
2022-10-16 14:21:19.390 [main]  DEBUG ktor.application - Watching C:\src\kotlin\ktor\gdayworld\build\classes\kotlin\main\plugins for changes.
2022-10-16 14:21:19.562 [main]  INFO  ktor.application - Application started in 0.298 seconds.
2022-10-16 14:21:19.562 [main]  INFO  ktor.application - Application started: io.ktor.server.application.Application@5a45133e
2022-10-16 14:21:19.937 [main]  INFO  ktor.application - Responding at http://127.0.0.1:8080
<===========--> 85% EXECUTING [15s]
> :run

When I change any source code now, the project rebuilds, and the app notices the recompiled classes, and restarts itself:

modified: C:\src\kotlin\ktor\gdayworld\src\main\kotlin\routes\IndexRoutes.kt
Change detected, executing build...


BUILD SUCCESSFUL in 6s
13 actionable tasks: 12 executed, 1 up-to-date

Waiting for changes to input files... (ctrl-d then enter to exit)
<=============> 100% EXECUTING [8m 55s]
2022-10-16 14:26:20.438 [eventLoopGroupProxy-4-2]  INFO  ktor.application - 200 OK: GET - /
2022-10-16 14:26:34.861 [eventLoopGroupProxy-3-1]  INFO  ktor.application - Changes in application detected.
2022-10-16 14:26:35.073 [eventLoopGroupProxy-3-1]  DEBUG ktor.application - Changes to 18 files caused application restart.
[...]
2022-10-16 14:26:35.106 [eventLoopGroupProxy-3-1]  INFO  ktor.application - Application auto-reloaded in 0.012 seconds.
2022-10-16 14:26:35.106 [eventLoopGroupProxy-3-1]  INFO  ktor.application - Application started: io.ktor.server.application.Application@33747fec
2022-10-16 14:26:35.107 [eventLoopGroupProxy-4-2]  INFO  ktor.application - 200 OK: GET - /
<===========--> 85% EXECUTING [5m 32s]
> :run

Note how the app doesn't restart until I actually use it, which is good thinking.

One might as why I have dropped down to a shell to do this autoload stuff? As far as I can tell it's not baked into IntelliJ yet, so needs to be handled directly by Gradle for now. It's not a hardship. I mean: the shells I am running there are being run from within IntelliJ, it's just slightly more complicated than a key-combo or some mouseclicks.

OK. That's all good progress. I'm gonna take a break and come back and create my own controller / response / etc, which is what the object of the exercise was today.


Docker

Ktor's way

I was not expecting this to be the next step, but I just spotted some stuff about Docker in the Ktor docs ("Docker"), so I decided to see what they said.

[time passes whilst I do battle with the docs]

OK, screw that. It's a very perfunctory handling of it. I can build a jar and create an image that will run it, and then run the container - and it all works - but it's… a bit… "proof of concept". From reading the docs and the code snippets that link from the docs (Deployment - Ktor plugin › Build and run a Docker image).

I had to add this to my build.gradle.kts file:

ktor {
    fatJar {
        archiveFileName.set("gday-world-ktor.jar")
    }
    docker {
        jreVersion.set(io.ktor.plugin.features.JreVersion.JRE_17)
        localImageName.set("gday-world-ktor")
        imageTag.set("${project.version}-preview")
        portMappings.set(listOf(
            io.ktor.plugin.features.DockerPortMapping(
                8080,
                8080,
                io.ktor.plugin.features.DockerPortMappingProtocol.TCP
            )
        ))
    }
}

And then from the shell I could run this lot:

PS C:\src\kotlin\ktor\gdayworld> ./gradlew :buildFatJar     
[…]
PS C:\src\kotlin\ktor\gdayworld> ./gradlew :runDocker

And I would indeed end up with a running Docker container. Which is handy, but I had no control over what params were passed to docker run, so I couldn't even give the container a name, so I just ended up with one of Docker's random ones. That's a bit amateurish. I checked to see if I was missing anything with the plugin, but this is the code (from Ktor's repo on GitHub):

private abstract class RunDockerTask : DefaultTask() {
    @get:Inject
    abstract val execOperations: ExecOperations

    @get:Input
    abstract val fullImageName: Property<String>

    @TaskAction
    fun execute() {
        val dockerExtension = project.getKtorExtension<DockerExtension>()
        execOperations.exec {
            it.commandLine(buildList {
                add("docker")
                add("run")
                for (portMapping in dockerExtension.portMappings.get()) {
                    add("-p")
                    with(portMapping) {
                        add("${outsideDocker}:${insideDocker}/${protocol.name.lowercase()}")
                    }
                }
                add(fullImageName.get())
            })
        }
    }
}

It looks to me like it simply builds a string docker run [port mappings] [image name], and that's it. No scope for me to specify any other of docker run's parameters in my build config.

So: nah, not doing that; I'll DIY. It's at least shown me what I need to do in a DockerFile and I can organise my own docker-compose.yml file.


My way

[…]

I have this docker/Dockerfile:

FROM gradle:7-jdk17 AS build
COPY --chown=gradle:gradle .. /home/gradle/src
WORKDIR /home/gradle/src
RUN gradle test --no-daemon
RUN gradle buildFatJar --no-daemon

FROM openjdk:17
EXPOSE 8080:8080
RUN mkdir /app
COPY --from=build /home/gradle/src/build/libs/*.jar /app/gday-world-ktor.jar
ENTRYPOINT ["java","-jar","/app/gday-world-ktor.jar"]

This is pretty much lifted from the Ktor Docker › Prepare Docker image docs I linked to above, I've just added the test-round in first.

And this docker/docker-compose.yml:

version: '3'

services:
  gday-world-ktor:
    build:
      context: ..
      dockerfile: docker/Dockerfile
    ports:
      - "8080:8080"
    stdin_open: true
    tty: true

And when I run docker-compose up --build --detach, after a couple of minutes, I get an up and running container with my app in it. Bonus: it halts if my tests don't first pass.

I'm not enamoured with the "after a couple of minutes" part of this: seems really slow for what it needs to do. I am "sure" there must be a way of telling Gradle to do the build, test-run and jar-build all on one operation. However I'm over googling things starting with "gradle" today, so I'm gonna leave it for now.

I'm pretty happy with the progress I made today.

Righto.

--
Adam

Sunday 9 October 2022

Kotlin / TDD: writing the tests for a small web service

G'day:

One of my tasks @ work is to check out how to test a web service. I started with this yesterday's article: Kotlin: getting the khttp library installed and running… then… getting rid of it and using something else, but that was justa "proof of concept" of making an HTTP call, and examining its results. Today I'm gonna write the actual tests we need for the interface for a web service. Note: I still don't know enough about Ktor to create a web service with it, so I'm gonna fall back to using CFML for that end of things. I'm not gonna bother with the code for that here; the tests will demonstrate whether or not the web service is fulfilling its contract. But I will be TDDing this. Currently I have zero tests and zero code to test. Let's get on with it.


It should return a 200-OK on the root URI on a valid request

fun `It should return a 200-OK on the root URI on a valid request`() {
    runBlocking {
        HttpClient().use { client ->
            val response = client.get(webserviceUrl)
            response.status shouldBe HttpStatusCode.OK
        }
    }
}

NB: I will only comment if there's something note-worthy or not obvious. All tests will initially fail until I implement the relevant bit of the web service, eg:

expected:<200 OK> but was:<404 Not Found>
Expected :200 OK
Actual   :404 Not Found

It should return a 406-NOT-ACCEPTABLE and suggest the expected type if the Accepts header is not application/json

@Test
fun `It should return a 406-NOT-ACCEPTABLE and suggest the expected type if the Accepts header is not application-json`() {
    runBlocking {
        HttpClient().use { client ->
            val response = client.get(webserviceUrl) {
                header("Accept", "text/plain")
            }

            response.status shouldBe HttpStatusCode.NotAcceptable
            response.body() as String shouldBe """["application/json"]"""
        }
    }
}

Note: It's a Kotlin/JVM limitation that I have to use application-json rather than application/json in the method name there. See Why Kotlin does not allow slash in identifiers, which in turn points the reader to Java Virtual Machine Specification › Chapter 4. The class File Format:

4.2.2. Unqualified Names

Names of methods, fields, local variables, and formal parameters are stored as unqualified names. An unqualified name must contain at least one Unicode code point and must not contain any of the ASCII characters . ; [ / (that is, period or semicolon or left square bracket or forward slash).

When I initially just referenced response.body() shouldBe """["application/json"]""" Kotlin was saying "Not enough information to infer type variable", and after some googling I landed on Type checks and casts › "Unsafe" cast operator, which explains that as String thing I have there.

Also note I needed to update the 200-OK test to pass the correct Accept header.


It returns an array of Numbers as a JSON array

This one was a bit trickier, but the docs were reasonably helpful, and I'm pleased with the outcome.

@Test
fun `It returns an array of Numbers as a JSON array`() {
    @Serializable
    data class Number(val id: Int, val en: String, val mi: String)

    runBlocking {
        HttpClient() {
            install(ContentNegotiation) {
                json()
            }
        }.use { client ->
            val response = client.get(webserviceUrl) {
                header("Accept", "application/json")
            }
            response.status shouldBe HttpStatusCode.OK
            response.body() as List<Number> shouldBe listOf(
                Number(1, "one", "tahi"),
                Number(2, "two", "rua"),
                Number(3, "three", "toru"),
                Number(4, "four", "wha")
            )
        }
    }
}

I had guessed that there'd be a way to deliver objects straight to the app from an HTTP request, and a quick google set me on the right path here, landing me on these docs: Content negotiation and serialization. What's super helpful in these docs as they link through to examples of everything (including the build.gradle.kt file, as I needed to add some dependencies - see further down), eg: ktor-documentation/codeSnippets/snippets/client-json-kotlinx/src/main/kotlin/com/example/Application.kt and ktor-documentation/codeSnippets/snippets/client-json-kotlinx/build.gradle.kts. I just followed those and changed the bits I needed to change.

Steps:

  • tell the client that I'm gonna expect it to work out what the content represents (boilerplate).
  • Create a data class that the client will use to deserialize the data as. NB: it needs to be tagged as being serializable, because well: that's what we're doing here.
  • Specify that type - or in this case a list of that type - as the body value.

Done.

I will admit than initially I thought I had messed-up because instead of getting a "nup, it's not a list of Numbers" in my failing test, I got this monstrousity:

io.ktor.client.call.NoTransformationFoundException: No transformation found: class io.ktor.utils.io.ByteBufferChannel -> class kotlin.collections.List

But it turns out that just means "um… yer request ain't returning JSON". Which it indeed was not. Once I got it to return something (anything) and the correct content-type, I got a more on-point error:

Expected start of the array '[', but had 'EOF' instead at path: $
JSON input: ""  

And from there I tweaked the web service to return a coupla wrong things to see how Ktor reported on deserialization failures, and it was all helpful.

All the examples I saw for this was only deserializing one object, so I was slightly cautious as to how to deal with a JSON array, but I took a punt on just specifying response.body() as List<Number>, thinking Ktor's probably clever enough to expect this sort of thing, and - lo - it did. Nice one.

As I mentioned above, I also had to tweak the dependencies and plugins a bit (build.gradle.kts):

plugins {
    kotlin("jvm") version "1.7.10"
    kotlin("plugin.serialization").version("1.7.10")
    application
}

// …

dependencies {
    // …
    implementation("io.ktor:ktor-client-core:$ktorVersion")
    implementation("io.ktor:ktor-client-cio:$ktorVersion")
    implementation("io.ktor:ktor-client-content-negotiation:$ktorVersion")
    implementation("io.ktor:ktor-serialization-kotlinx-json:$ktorVersion")
    // …
}

It will accept a POST request of an object as JSON and return the same object as confirmation, and its URL

This was really straight-forward, having done all the hard-bit in the previous one:

@Test
fun `It will accept a POST request of an object as JSON and return the same object as confirmation, and its URL`() {
    val five = Number(5, "five", "rima")
    runBlocking {
        HttpClient() {
            install(ContentNegotiation) {
                json()
            }
        }.use { client ->
            val response = client.post(webserviceUrl) {
                contentType(ContentType.Application.Json)
                setBody(five)
            }
            response.status shouldBe HttpStatusCode.Created
            response.body() as Number shouldBe five
            response.headers["Location"] shouldBe "${webserviceUrl}5"
        }
    }
}

By now there's absolutely nothing "unexpected" in this, I think.


OK that's enough for a Sunday afternoon. This weekend I've managed to work out how to make HTTP requests in my tests, how to set/check headers, response codes and the body of the responses. I've posted an object and received objects back again, letting Ktor handle the (de)serialization.

These tests are only testing the interface of the web webservice, which is fine and an essential part of building a web service. However the next thing on the list is to do end to end tests: check the underlying data store that new objects are being created (cos they absolutely are not at the moment ;-)), and the correct data is being returned, etc. I have no idea how to make a DB call in Kotlin yet. Something for the coming week, I guess.

The code is on GitHub @ /src/test/kotlin/junit/practical/WebServiceTest.kt, and the web stub service I was creating to make the tests pass is this lot: adamcameron/Numbers.cfc.

Righto.

--
Adam

Friday 7 October 2022

Kotlin: getting the khttp library installed and running... then... getting rid of it and using something else

G'day:

So 50% of this was a largely fruitless exercise in the end. Other than the fact that I learned some stuff that I think is worth knowing for future reference.

The next thing on my Kotlin/testing list is a more practical exercise: I need to be able to write tests for web service endpoints. To do that I need to be able to make HTTP requests. I second-guessed there'd be really awful boilerplate-laiden Java ways of doing things, but I wanted a Kotlin-idiomatic way of doing it.


khttp

I googled "kotlin http requests", and one of the first links was to an article on the Baeldung website: "HTTP Requests with Kotlin and khttp". Their articles ae usually pretty solid, so I didn't look any further. It was especially appealing because they made a point of saying:

On the JVM we have several available options, from lower-level to very high-level libraries[…]. However, most of them are targeted primarily at Java programs.

In this article, we’re going to look at khttp, an idiomatic Kotlin library for consuming HTTP-based resources and APIs.

Perfect.

First I had to install this khttp thing. The instructions were for a hand-cranked Maven install, but I can't be arsed with that at the moment, and I wanna use Gradle via my build.gradle.kts file. Other things I've looked at have supplied the installation instructions for that, which is really handy for a n00b like me as I don't really know what I'm doing and what the various values mean in the Maven XML. And TBH: I don't want to know. But I figured I could work out from the XML:

<dependency>
    <groupId>khttp</groupId>
    <artifactId>khttp</artifactId>
    <version>0.1.0</version>
</dependency>

<repository>
    <id>central</id>
    <url>http://jcenter.bintray.com</url>
</repository>

I need the repository cos it's not on Maven Central yet apparently. Cool: I know that one can specify repositories and dependencies in the Gradle file, so how hard could it be.

repositories {
    mavenCentral()
    maven("https://jcenter.bintray.com")
}

dependencies {
    // …
    testImplementation("khttp:khttp:0.1.0")
}

All I needed to do is to change the repo's scheme from http to https, because with http I was getting an error:

Using insecure protocols with repositories, without explicit opt-in, is unsupported.

And that's fair enough. It installed OK over https.

I created my test:

@DisplayName("Tests of WebService class")
internal class WebServiceTest {

    val serviceUrl = "https://example.com/"
    @Test
    fun `it returns a 200`() {
        val response = khttp.get(serviceUrl)

        response.statusCode shouldBe 200
    }
}

And ran it:

Unable to make field private java.lang.String java.net.URL.host accessible: module java.base does not "opens java.net" to unnamed module @645aa696 java.lang.reflect.InaccessibleObjectException: Unable to make field private java.lang.String java.net.URL.host accessible: module java.base does not "opens java.net" to unnamed module @645aa696 at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)

Not what I wanted to see. Not least of all cos I have NFI what that means. I poss should have noticed the mention of reflection in which case I might have recalled that there's been warnings emiting from Java about this when I start-up Lucee:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by [x] to method [y]
WARNING: Please consider reporting this to the maintainers of [x]
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

Had I thought about it I would have landed on the fix sooner. It took me about an hour to find a comment on the khttp projects GitHub issues: ascclemens/khttp issue #88. I am running on a Java 16 JDK, and that reflective thing got blocked from that version on. The best answer on the issue was:

For enabling khttp 1.0 with JDK 17.0 I use these VM arguments:

--add-opens java.base/java.net=ALL.UNNAMED --add-opens java.base/sun.net.www.protocol.https=ALL-UNNAMED

Cool. I had a hunt around in the settings of IntelliJ a bit to find out where I could sling those. I found a few places that took VM arguments, which looked promising, but none worked when I rebuild the project. I also found a way of putting them in my build file:

application {
    mainClass.set("MainKt")
    listOf(
        "--add-opens java.base/java.net=ALL.UNNAMED",
        "--add-opens java.base/sun.net.www.protocol.https=ALL-UNNAMED"
    ).also { applicationDefaultJvmArgs = it }
}

Which is good to know, but also didn't work.

In the end I downloaded Java 15 JDK and used that instead. For future me, the setting that matter is the Gradle JVM:

It is not just a matter of changing the Java or Kotlin byte-code versions:

I could switch those to 15, and it didn't make a difference. But once I changed Gradles JVM to 15, it worked:

But do you know what? I wasn't so happy with reverting my JVM version to accommodate a library I was using for testing, so I decided to see if there was another one. And this is when I wished I had looked below the fold on the first page of Google results…


Ktor

Ktor is the framework we have decided to use for our project @ work, so it kinda makes sense to use its own tooling. I'm gonna have a look at their offering.

Installation was simple:

implementation("io.ktor:ktor-client-core:2.1.2")
implementation("io.ktor:ktor-client-cio:2.1.2")

The first one is self-explanatory; the second is the underlying engine the client uses it seems. There's some docs: Ktor Client / Developing applications / Setting up a client / Engines.

From there the test was a bit of horsing around:

fun `it returns a 200`() {
    runBlocking {
        HttpClient().use { client ->
            val response = client.get("https://jsonplaceholder.typicode.com/todos/1")
            response.status shouldBe HttpStatusCode.OK
        }
    }
}

The CIO engine is asynchronous, so all this is using coroutines which I don't have headspace for ATM, but it seems the "async await" approach to making async code run in non-async code is to sling a runBlocking block around it. Fine.

One cool Kotlin thing here is that I did have this code written like this:

fun `it returns a 200`() {
    runBlocking {
        val client = HttpClient()
        try {
            val response = client.get("https://jsonplaceholder.typicode.com/todos/1")
            response.status shouldBe HttpStatusCode.OK
        } finally {
            client.close()
        }
    }
}

IntelliJ had a squiggly line under the try, and the hint said "try-finally can be replaced with 'use()'". Um: sure. Can it? OK. Let's do that then. And it changed it for me to the code I initially showed.

I was bemused that the client.close() call had gone completely. I RTFMed a bit, and use is quite cool:

Executes the given block function on this resource and then closes it down correctly whether an exception is thrown or not.

It can be called on any object that implements the Closeable interface (although use is a Kotlin extension to this). If one looks at the implementation of Closeable.use we see what's going on (this is heavily elided):

fun use() {
    try {
        return block(this)
    } finally {
        try {
            close()
        } catch (closeException: Throwable) {
        }
    }
}

This use approach just makes sense.


Thoughts

In contrasting the very simple khttp approach to the Ktor approach, I prefer the khttp one. This reminds me of GuzzleHttp for PHP: it started off being a simple and useful and just… did HTTP requests. No async, no promises, it just got on with it. Then it decided async was all the rage so it took a very simple interface and made it overly complex for the task at hand (not to mention not backwards compatible). Ktor's approach seems similar. When I reflect upon how often I really need my HTTP requests to be done asynchronously to my mainline code, the answer is fucking never. Or so infrequently then I'm happy to write an async wrapper for the call if I need it.

But: we're already using Ktor, and it's more mature, and I don't have to retrograde to an old version of Java to use it. So Ktor and its unecessary complexity it is.

I'll try to do something more useful with this stuff tomorrow, but getting everything working (twice) took more time than I wanted to spend on this, so I'm stopping here for now.

The code for this is on GitHub @ /src/test/kotlin/junit/practical/WebServiceTest.kt.

Righto.

--
Adam