<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>But Not In That Order</title>
    <description>Judson writes about a variety of things, mostly technical. Computer programming, systems administration, the culture and society around both.
</description>
    <link>http://judsonlester.info/</link>
    <atom:link href="http://judsonlester.info/feed.xml" rel="self" type="application/rss+xml"/>
    <pubDate>Mon, 10 Nov 2025 00:03:33 +0000</pubDate>
    <lastBuildDate>Mon, 10 Nov 2025 00:03:33 +0000</lastBuildDate>
    <generator>Jekyll v3.10.0</generator>
    
      <item>
        <title>inlinefiles</title>
        <description>&lt;p&gt;Say you’ve got a simple web-service to write.
It’s going to be a microservice,
and in order to
play nice in its environment,
you want it to serve a static file
with JSON that describes
where it’s monitoring page is
and who to get in touch with if it misbehaves.
You could
include a literal JSON string,
but it becomes harder to work with
than it would be in its own file.&lt;/p&gt;

&lt;p&gt;When the data is
in its own file,
your editor can highlight the syntax correctly.
It’s easier to check that the picky JSON syntax is correct.
JSON files are easier to use in different contexts
than Go string literals that are JSON,
or constant structs marshalled onto the wire.&lt;/p&gt;

&lt;p&gt;In Node or Ruby,
you can include the file itself
in the application package,
and refer to it relative to the
endpoint code.
In Ruby, it looks like&lt;/p&gt;
&lt;div class=&quot;language-ruby highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;nb&quot;&gt;puts&lt;/span&gt; &lt;span class=&quot;no&quot;&gt;File&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;read&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;no&quot;&gt;File&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;join&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;kp&quot;&gt;__FILE__&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;&apos;../metadata.json&apos;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;))&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;
&lt;h2 id=&quot;compiled-languages-though&quot;&gt;Compiled Languages Though&lt;/h2&gt;

&lt;p&gt;Coming to Go
from a dynamic languages background,
I got tripped up a bit by the fact that
your source code isn’t available
at runtime.
It’s obvious in retrospect,
but I was fully prepared to
use &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;os.Open&lt;/code&gt; and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;io.Write&lt;/code&gt;
to emit my JSON to the wire.
But after it’s compiled,
Go code doesn’t have direct access to its source code.&lt;/p&gt;

&lt;p&gt;So somehow we need to
incorporate the data files
into our source code.
We can translate them into strings,
and perform escaping so that regardless of what’s in the files,
they’re legal string literals.&lt;/p&gt;

&lt;p&gt;Go has &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;go:generate&lt;/code&gt;,
which is very handy for this.
If we include a magic comment in our code,
then running &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;go generate&lt;/code&gt; on the command line
will run commands to produce code.
There’s support in the stdlib
for generating &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;Stringer&lt;/code&gt; interfaces,
for instance.&lt;/p&gt;

&lt;p&gt;It would also be handy
if we could continue to treat the files
&lt;em&gt;as files&lt;/em&gt; instead of strings.
In part,
because this means that it’s easier to work
with template inclusion.
Also because it means that if we later want
to override the hard-coded files,
we’re still using an &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;os.File&lt;/code&gt; or &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;io.Buffer&lt;/code&gt;
instead of mixing those types with &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;string&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Go has the &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;vfs&lt;/code&gt; package.
It’s part of &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;go doc&lt;/code&gt;
but it serves our purposes perfectly.
You can define a &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;mapfs&lt;/code&gt;
which is a literal &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;map[string]string&lt;/code&gt;,
but allows you to &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;Open(path)&lt;/code&gt; and get file structs.&lt;/p&gt;

&lt;p&gt;So there’s a lot of facility to do what we want in go already.
All that’s missing is a tool to take
a real directory structure,
and generate the code for a literal map.&lt;/p&gt;

&lt;h2 id=&quot;but-wheres-the-shill&quot;&gt;But Where’s the Shill?&lt;/h2&gt;

&lt;p&gt;Here’s &lt;a href=&quot;https://github.com/nyarly/inlinefiles&quot;&gt;inlinefiles&lt;/a&gt; to weld all this together.
&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;inlinefiles&lt;/code&gt; takes two arguments:
a path to a directory of source files,
and a path to an output file,
and generates Go code to build an equivalent &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;mapfs&lt;/code&gt;.
There’s a few flags to handle things like
using a specific package name (instead of guessing)
and to restrict the files included based on a glob pattern.
It’s suitable for using as a &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;//go:generate&lt;/code&gt; comment
(since that was the motivating case.)&lt;/p&gt;

&lt;p&gt;In addition, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;inlinefiles&lt;/code&gt; includes a package for
smoothing out using &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;vfs.FileSystem&lt;/code&gt;s with
Go templates.
It seemed reasonable to add
because the problem is more significant
if you want to use templates.&lt;/p&gt;

&lt;p&gt;There’s more pressure to use
separate files for templates,
where the concerns about whitespace
for the template output
and for the code itself
are in tension.
Groups of templates are also coupled to one another
when one template includes another.
They need a file hierarchy provided to them,
in the form of paths to the other template files,
or links of some kind of the loaded templates themselves.
It gets a little more annoying to manage.&lt;/p&gt;

&lt;p&gt;So, while template handling is in tension with
“do one thing”
but it’s a common enough use case,
and the extra code is light enough.&lt;/p&gt;

&lt;p&gt;Candidly,
I only started working on &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;inlinefiles&lt;/code&gt;
after searching for alternatives
and reviewing a few existing packages.
(It’s my understanding that &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;packr&lt;/code&gt; is well thought of,
for instance.)
The thing that drove me to build my own
really was the desire for something small
and easy to understand.
The design intention was
to adhere to the Unix principle
of small tools doing one thing well.
Re-using as much of the Go core tooling
as I could
was an important goal.&lt;/p&gt;

&lt;h2 id=&quot;rewrite-it-in-rust&quot;&gt;Rewrite It in Rust!&lt;/h2&gt;

&lt;p&gt;You might’ve been asking yourself
“if compiled languages have this problem,
why isn’t there a Rust analogue of &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;inlinefiles&lt;/code&gt;?”
Which is a fair question,
but the answer is breathtakingly simple:
the Rust stdlib ships with a &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;include_bytes!&lt;/code&gt; macro
(and a couple of related ones)
that do exactly this: include the contents of a file
as a &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;&amp;amp;static [u8]&lt;/code&gt;,
which means that it all happens at compile time,
without needing an additional
(if quite excellent)
build tool.&lt;/p&gt;
</description>
        <pubDate>Sun, 05 May 2019 00:00:00 +0000</pubDate>
        <link>http://judsonlester.info/2019/05/05/inlinefiles/</link>
        <guid isPermaLink="true">http://judsonlester.info/2019/05/05/inlinefiles/</guid>
        
        <category>tool</category>
        
        <category>software</category>
        
        
      </item>
    
      <item>
        <title>dockerfiledeps</title>
        <description>&lt;p&gt;Somewhere along the line
(I’m not sure exactly where)
a significant part of my work life
became building Docker images.
Often,
clusters of related Docker images.
Specifically,
writing Makefiles
to build Docker images.&lt;/p&gt;

&lt;p&gt;A frustrating thing about this
is that Docker is extremely conservative
about what layers it will cache,
and it rebuilds all the layers that follow
a layer that changes.
When the reason
that I am writing Makefiles
is that I’m building closely related images,
they tend to have many layers
in common with each other,
but unique to the project.
Another reason I’m writing Makefiles
is to try to not rebuild things that
don’t need to be rebuilt.&lt;/p&gt;

&lt;p&gt;I like to
maintain a tight development cycle
for all my projects,
and the length of Docker builds
was getting to me.&lt;/p&gt;

&lt;p&gt;As it happens,
I’ve elsewhere written Go code
against the Docker packages
that parse the Dockerfile format.
It turns out this is relatively simple,
so it was easy to build a utility
that could parse a Dockerfile
and produce a Makefile inclusion
to handle the image’s dependencies
on the files it adds
and the image it descends from.
Effectively,
we want &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;gcc -M&lt;/code&gt;
but for &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;docker build&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;dockerfiledeps&lt;/code&gt;
is that tool.
Here’s how it works:&lt;/p&gt;

&lt;p&gt;First,
install &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;dockerfiledeps&lt;/code&gt;
with the usual Go incantation:
&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;go get github.com/nyarly/dockerfiledeps&lt;/code&gt;.
Then,
run &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;dockerfiledeps -emit-driver &amp;gt; driver.mk&lt;/code&gt;
and start a &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;Makefile&lt;/code&gt;
like&lt;/p&gt;
&lt;div class=&quot;language-make highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;nv&quot;&gt;REGISTRY_HOST&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;:=&lt;/span&gt; my.registry.net
&lt;span class=&quot;nv&quot;&gt;REPOSITORY_NAME&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;:=&lt;/span&gt; project
&lt;span class=&quot;k&quot;&gt;include&lt;/span&gt;&lt;span class=&quot;sx&quot;&gt; driver.mk&lt;/span&gt;

&lt;span class=&quot;nl&quot;&gt;push-all&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;push-foo&lt;/span&gt;
  &lt;span class=&quot;err&quot;&gt;@echo&lt;/span&gt; &lt;span class=&quot;err&quot;&gt;done&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;You create your project repo
with directories for each target image,
where the directory name is the sub-repo name of the image.&lt;/p&gt;

&lt;p&gt;When image &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;foo&lt;/code&gt; descends from image &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;bar&lt;/code&gt;,
start its Dockerfile
with &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;FROM my.registry.net/project/bar:local&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Then you can &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;make push-all&lt;/code&gt; and
get the Make goodness of minimal build times.
You can add rules for files in the various image directories
to speed builds, as well.
After all, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;dockerfiledeps&lt;/code&gt; is intended
to help and streamline Makefile writing,
not to completely replace it.&lt;/p&gt;

&lt;p&gt;There are a number of
somewhat opinionated
but really helpful things
that &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;dockerfiledeps&lt;/code&gt; also does.&lt;/p&gt;

&lt;p&gt;The &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;dockerfiledeps&lt;/code&gt; driver file
creates file proxy targets
(or proxy file targets…)
for a number of build steps.
For instance,
the &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;docker pull&lt;/code&gt; steps
capture the resulting Docker id
into a file,
which gets updated if it changes.
The upshot is that you can depend
on these “proxy files”
and rely on Make tasks to run
if their inputs have changed,
but only then.&lt;/p&gt;

&lt;p&gt;It captures git status data
and uses it both for Docker labelling purposes
(e.g. it uses the most recent git tag as a label)
and as &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;--build-arg&lt;/code&gt;s to the Dockerfile.
You could, if you wanted
have your Dockerfile bail out
if the git repository isn’t clean,
or simply record the clean status
as a Docker label.&lt;/p&gt;

&lt;p&gt;If you have &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;pv&lt;/code&gt; installed,
the output of Docker will
get piped through it,
so you’ll have some sense
that the build is proceeding
without the Tank effect.&lt;/p&gt;

&lt;p&gt;Keep in mind that the &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;driver.mk&lt;/code&gt; file
is intended as a quick-start.
You can absolutely update it;
that’s part of why
the &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;-emit-driver&lt;/code&gt; mode
doesn’t write to a file by default.
It’s the &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;gcc -M&lt;/code&gt; behavior
that we’re really after,
and the driver file
is first and foremost there
to support using that.
Don’t be afraid to
bring it up in an editor and change it.
This is your Makefile,
after all.&lt;/p&gt;

&lt;p&gt;Please do let me know
if you get use out of dockerfiledeps,
or if you run across some deficiency.
Ultimately, because of how Make and Docker interact,
I’m almost sure there are situations
without an ideal solution.
They should all be fixable with Make rules,
though.&lt;/p&gt;
</description>
        <pubDate>Fri, 14 Dec 2018 00:00:00 +0000</pubDate>
        <link>http://judsonlester.info/2018/12/14/dockerfiledeps/</link>
        <guid isPermaLink="true">http://judsonlester.info/2018/12/14/dockerfiledeps/</guid>
        
        <category>tool</category>
        
        <category>software</category>
        
        
      </item>
    
      <item>
        <title>Simple Webservers with NixOps</title>
        <description>&lt;p&gt;After working with Nix and NixOS casually for about a year
I started to play with NixOps, the deployment tooling built on top of Nix.
Right away, my reaction was
“gosh, that was easy!”&lt;/p&gt;

&lt;p&gt;This is such a rare experience
with any software,
much less anything operations related,
it was surprising.
The process was so straightforward
and rapid
that I wanted to share the details.
Hopefully this’ll serve as
a quick introduction to the Nix ecosystem,
and to NixOps in particular.&lt;/p&gt;

&lt;h2 id=&quot;what-had-happened-was&quot;&gt;What Had Happened Was…&lt;/h2&gt;

&lt;p&gt;For much of the last decade,
I was a co-owner of
a full service web consultancy.
Operations tasks were
primarily my responsibility.
A little bit over two years ago,
the consultancy folded
(which is a story for another time).
I’m finally getting around
to the process of rejiggering
legacy servers there,
mostly to reduce operating costs.&lt;/p&gt;

&lt;p&gt;One of the services hosted there
was this blog.
For a variety of reasons,
I wanted to migrate my personal web presence
onto servers I’m directly responsible for.&lt;/p&gt;

&lt;p&gt;The last time I refreshed my working machine,
I converted it over to &lt;a href=&quot;https://nixos.org/&quot;&gt;NixOS&lt;/a&gt;,
and I’ve been so pleased
with that ongoing experience
that I thought I’d give NixOps
a try for moving my stuff
onto servers
that I run out of pocket.&lt;/p&gt;

&lt;h2 id=&quot;local-work&quot;&gt;Local Work&lt;/h2&gt;

&lt;p&gt;I use &lt;a href=&quot;https://jekyllrb.com/&quot;&gt;Jekyll&lt;/a&gt; for this blog;
I probably sorely under-use it.
It’s a very solid and powerful static site generator,
and Ruby remains well inside my comfort zone.&lt;/p&gt;

&lt;p&gt;But rather than worry about
which version of Ruby I have installed,
(and if libxml2 has slipped with respect to mini_portile2…)
I use &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;nix-shell&lt;/code&gt; to set up my working environment.
Without going into too much detail,
it works something like &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;rvm&lt;/code&gt;
(or &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;rbenv&lt;/code&gt; or &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;virtualenv&lt;/code&gt; or &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;nodenv&lt;/code&gt; or …)
in a language-platform agnostic way.&lt;/p&gt;

&lt;p&gt;Again, this is probably overkill for a blog,
but for my development projects it’s been fantastic.
I especially enjoy that
I can use the same tool
irrespective of
language or required dependencies.
For instance,
I use &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;nix-shell&lt;/code&gt;
with a large Go project that
has dependencies on PostgreSQL and Java.
I use it with
Rust and
to develop Ruby gems.
The only thing better than
using &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;nix-shell&lt;/code&gt; myself
would be working on a team that
all used &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;nix-shell&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The core model within Nix
is an “expression” that describes
how to build a piece of software.
It works much like
a package definition in
any other OS distribution system.
Expressions are written in
the Nix programming language,
which makes them very flexible.&lt;/p&gt;

&lt;p&gt;One example of that flexibility is that
&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;nix-shell&lt;/code&gt; can take the expression
writen for building a software package
and set up a shell environment with
all of its build-time dependencies available.
The primary motivation
for this workflow is
to work smoothly with packages
as you prepare them for distribution.&lt;/p&gt;

&lt;p&gt;But it also works tremendously well
for the original development of the project.
I’ve been treating this blog
as an ongoing software project,
and using an expression to describe
its “build” environment.
That meant picking a version of Ruby,
installing Jekyll and Rake
and plugins
and all of their dependencies,
and keeping those installs isolated
from other Ruby work I do
on the same system.&lt;/p&gt;

&lt;p&gt;This blog’s build expression lives
in a file called
(by Nix convention)
&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;default.nix&lt;/code&gt;
and it looks like this:&lt;/p&gt;
&lt;div class=&quot;language-nix highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;pkgs&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;?&lt;/span&gt; &lt;span class=&quot;kr&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;nixpkgs&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{}&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;}:&lt;/span&gt;
&lt;span class=&quot;kd&quot;&gt;let&lt;/span&gt;
  &lt;span class=&quot;kn&quot;&gt;inherit&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;pkgs&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;lib&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;stdenv&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;ruby&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;bundler&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;bundlerEnv&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;fetchFromGitHub&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;

  &lt;span class=&quot;c&quot;&gt;# This sets up the Ruby environment, complete with Bundler&apos;s Gemfile&lt;/span&gt;
  &lt;span class=&quot;c&quot;&gt;# Many of these settings have defaults, but I tend towards the explicit.&lt;/span&gt;
  &lt;span class=&quot;nv&quot;&gt;rubyEnv&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;bundlerEnv&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;kn&quot;&gt;inherit&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;ruby&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;nv&quot;&gt;name&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;jekyll-blog&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;nv&quot;&gt;gemfile&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;sx&quot;&gt;./Gemfile&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;nv&quot;&gt;lockfile&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;sx&quot;&gt;./Gemfile.lock&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;c&quot;&gt;# This file is specially generated to adapt Rubygems to the Nixpkgs distribution&lt;/span&gt;
    &lt;span class=&quot;nv&quot;&gt;gemset&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;sx&quot;&gt;./gemset.nix&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
  &lt;span class=&quot;p&quot;&gt;};&lt;/span&gt;
&lt;span class=&quot;kn&quot;&gt;in&lt;/span&gt;
  &lt;span class=&quot;nv&quot;&gt;stdenv&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;mkDerivation&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;nv&quot;&gt;name&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;blog-jdl&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;c&quot;&gt;# This conditional means that I work with the local source for authoring,&lt;/span&gt;
    &lt;span class=&quot;c&quot;&gt;# but pin a particular git revision for publication.&lt;/span&gt;
    &lt;span class=&quot;nv&quot;&gt;src&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;kr&quot;&gt;builtins&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;pathExists&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;sx&quot;&gt;./source.nix&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;then&lt;/span&gt;
      &lt;span class=&quot;nv&quot;&gt;fetchFromGitHub&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;nv&quot;&gt;owner&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;nyarly&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;nv&quot;&gt;repo&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;blog&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;nv&quot;&gt;rev&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;master&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;nv&quot;&gt;sha256&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;kr&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;sx&quot;&gt;./source.nix&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;else&lt;/span&gt;
      &lt;span class=&quot;sx&quot;&gt;./.&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;nv&quot;&gt;buildInputs&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;
      &lt;span class=&quot;nv&quot;&gt;rubyEnv&lt;/span&gt;
      &lt;span class=&quot;nv&quot;&gt;bundler&lt;/span&gt;
    &lt;span class=&quot;p&quot;&gt;];&lt;/span&gt;

    &lt;span class=&quot;c&quot;&gt;# These are the steps to actually build and publish the blog.&lt;/span&gt;
    &lt;span class=&quot;c&quot;&gt;# nix-shell ignores them - I just run `jekyll serve` to author.&lt;/span&gt;
    &lt;span class=&quot;nv&quot;&gt;buildPhase&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;jekyll build&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;nv&quot;&gt;installPhase&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;cp -a _site $out&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
  &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This is the whole
&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;Rakefile&lt;/code&gt;
for this blog.
I could probably get away with
a single “publish.rb”
(or a shell script.)&lt;/p&gt;

&lt;div class=&quot;language-ruby highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;task&lt;/span&gt; &lt;span class=&quot;ss&quot;&gt;:update&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;do&lt;/span&gt;
  &lt;span class=&quot;c1&quot;&gt;# This command captures the remote git revision into a file for deployment.&lt;/span&gt;
  &lt;span class=&quot;c1&quot;&gt;# Note that this is the file that triggers the src conditional in default.nix.&lt;/span&gt;
  &lt;span class=&quot;n&quot;&gt;sh&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;nix-prefetch-git --no-deepClone git@github.com:nyarly/blog.git| jq &apos;.sha256&apos; &amp;gt; &lt;/span&gt;&lt;span class=&quot;si&quot;&gt;#{&lt;/span&gt;&lt;span class=&quot;no&quot;&gt;ENV&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;s1&quot;&gt;&apos;NIXOPS_DIR&apos;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/blog/source.nix&quot;&lt;/span&gt;

  &lt;span class=&quot;c1&quot;&gt;# Then we keep the build description files in sync with the deployment configuration.&lt;/span&gt;
  &lt;span class=&quot;sx&quot;&gt;%w[default.nix Gemfile Gemfile.lock gemset.nix]&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;each&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;do&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;|&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;file&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;|&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;cp&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;file&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;no&quot;&gt;Pathname&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;new&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;no&quot;&gt;ENV&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;s1&quot;&gt;&apos;NIXOPS_DIR&apos;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]).&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;join&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;blog&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;file&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;).&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;expand_path&lt;/span&gt;
  &lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;from-my-desk-to-your-screen&quot;&gt;From My Desk to Your Screen&lt;/h2&gt;

&lt;p&gt;Most of the above
is serves as a kind of prologue.
I’d already built it up
as I got more confident with using
Nix with Ruby and Jekyll.
There some adaptation to make it work with deployment,
but interestingly,
none of those adaptations
are &lt;em&gt;commitments.&lt;/em&gt;
I could go back to rsyncing to Nginx
(or converting the whole thing to S3)
without having to undo my changes.&lt;/p&gt;

&lt;p&gt;What was really cool
was that I got from nothing
(literally, an empty console on AWS)
to a running server
in 3 20-minute sessions.
The hardest thing
(as ever)
was deciding whether to use
&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;mod_rewrite&lt;/code&gt; or &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;mod_redirect&lt;/code&gt;
in Apache.&lt;/p&gt;

&lt;p&gt;In NixOS, all the packaging is
handled by Nixpkgs,
which is the Nix based package distribution.
Configuration is abstracted into
a flavor of expression called a “module”,
and modules are composed into
a single system-wide configuration.
The consequence is that you wind up
writing an expression to build
a computer installation
in the same way that you describe
the build and installation of
a piece of software.
In a manner of speaking,
you build a computer the same way
that you build a piece of software.&lt;/p&gt;

&lt;p&gt;NixOps is the
deployment and orchestration tooling
built on top of NixOS.
You arrange computer configurations
into networks,
and pair them with descriptions of
how to provision hardware for them.&lt;/p&gt;

&lt;p&gt;This is how
a simple Apache webserver
configured to work with &lt;a href=&quot;https://letsencrypt.org/&quot;&gt;LetsEncrypt&lt;/a&gt; looks:&lt;/p&gt;

&lt;div class=&quot;language-nix highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;kd&quot;&gt;let&lt;/span&gt;
  &lt;span class=&quot;c&quot;&gt;# This pulls in the default.nix for the blog,&lt;/span&gt;
  &lt;span class=&quot;c&quot;&gt;# as copied into the deployment repo by the Rakefile.&lt;/span&gt;
  &lt;span class=&quot;nv&quot;&gt;blog&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;kr&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;sx&quot;&gt;./blog/default.nix&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{};&lt;/span&gt;
  &lt;span class=&quot;nv&quot;&gt;acmeRoot&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;/var/run/acme-challenges&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;kn&quot;&gt;in&lt;/span&gt;
  &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;nv&quot;&gt;network&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;description&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;Web server&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;nv&quot;&gt;webserver&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;
      &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;...&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;}:&lt;/span&gt;
      &lt;span class=&quot;c&quot;&gt;# This is the server configuration as it would be provided to NixOS if&lt;/span&gt;
      &lt;span class=&quot;c&quot;&gt;# I were deploying by hand.&lt;/span&gt;
      &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;nv&quot;&gt;services&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;httpd&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
          &lt;span class=&quot;nv&quot;&gt;enable&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;kc&quot;&gt;true&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
          &lt;span class=&quot;nv&quot;&gt;adminAddr&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;nyarly@gmail.com&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
          &lt;span class=&quot;nv&quot;&gt;extraConfig&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&apos;&apos;&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;
&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;            &amp;lt;Directory &lt;/span&gt;&lt;span class=&quot;si&quot;&gt;${&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;acmeRoot&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/judsonlester.info&amp;gt;&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;
&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;              Require all granted&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;
&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;            &amp;lt;/Directory&amp;gt;&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;
&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;          &apos;&apos;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
          &lt;span class=&quot;nv&quot;&gt;virtualHosts&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;
            &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
              &lt;span class=&quot;nv&quot;&gt;hostName&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;judsonlester.info&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
              &lt;span class=&quot;nv&quot;&gt;serverAliases&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;www.judsonlester.info&quot;&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;];&lt;/span&gt;
              &lt;span class=&quot;nv&quot;&gt;listen&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[{&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;port&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;80&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;}];&lt;/span&gt;
              &lt;span class=&quot;nv&quot;&gt;extraConfig&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&apos;&apos;&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;
&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;                Redirect / https://judsonlester.info&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;
&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;                Alias &quot;/.well-known/acme-challenge&quot; &quot;&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;${&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;acmeRoot&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/judsonlester.info/.well-known/acme-challenge&quot;&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;
&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;              &apos;&apos;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;

            &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
            &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
              &lt;span class=&quot;nv&quot;&gt;hostName&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;judsonlester.info&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
              &lt;span class=&quot;nv&quot;&gt;serverAliases&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;www.judsonlester.info&quot;&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;];&lt;/span&gt;
              &lt;span class=&quot;nv&quot;&gt;listen&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[{&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;port&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;443&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;}];&lt;/span&gt;

              &lt;span class=&quot;c&quot;&gt;# This is where the blog itself gets actually served.&lt;/span&gt;
              &lt;span class=&quot;nv&quot;&gt;documentRoot&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;blog&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
              &lt;span class=&quot;nv&quot;&gt;enableSSL&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;kc&quot;&gt;true&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
              &lt;span class=&quot;nv&quot;&gt;sslServerCert&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;/var/lib/acme/judsonlester.info/full.pem&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
              &lt;span class=&quot;nv&quot;&gt;sslServerKey&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;/var/lib/acme/judsonlester.info/key.pem&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;

              &lt;span class=&quot;nv&quot;&gt;extraConfig&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&apos;&apos;&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;
&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;                Alias &quot;/.well-known/acme-challenge&quot; &quot;&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;${&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;acmeRoot&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/judsonlester.info/.well-known/acme-challenge&quot;&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;
&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;              &apos;&apos;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
            &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
          &lt;span class=&quot;p&quot;&gt;];&lt;/span&gt;
        &lt;span class=&quot;p&quot;&gt;};&lt;/span&gt;

        &lt;span class=&quot;nv&quot;&gt;security&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;acme&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;certs&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
          &lt;span class=&quot;s2&quot;&gt;&quot;judsonlester.info&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
            &lt;span class=&quot;nv&quot;&gt;webroot&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;acmeRoot&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;/judsonlester.info&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
            &lt;span class=&quot;nv&quot;&gt;email&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;nyarly@gmail.com&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
          &lt;span class=&quot;p&quot;&gt;};&lt;/span&gt;
        &lt;span class=&quot;p&quot;&gt;};&lt;/span&gt;

        &lt;span class=&quot;nv&quot;&gt;networking&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;firewall&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;allowedTCPPorts&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;22&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;80&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;443&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;];&lt;/span&gt;
      &lt;span class=&quot;p&quot;&gt;};&lt;/span&gt;
  &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;If you’re not familiar with setting up Apache,
don’t worry too much.
It’s mostly pretty tedious,
and you can basically fiddle around
with the documentation until it does what you want.
With &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;nix-shell&lt;/code&gt; it’s easy to
set up an Apache server
and experiment locally
until you have configs that are working!&lt;/p&gt;

&lt;p&gt;If you are familiar with this process,
I hope you’ll agree that &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;webserver.nix&lt;/code&gt;
describes just what’s unique about this server.
The rigmarole of installing packages
and maintaining configurations
is all reduced to
what amounts to
expressing how what you want
varies from a stock install.
Having specified
&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;services.httpd.enable = true&lt;/code&gt;
(i.e. “install and turn up Apache”),
much of the configuration
addresses how I want that to vary
from a basic Apache installation.&lt;/p&gt;

&lt;p&gt;There’s also an &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;ec2.nix&lt;/code&gt; that describes
the AWS EC2 instances I wanted to deploy to,
the security groups to apply,
IP addresses,
and all that stuff.&lt;/p&gt;

&lt;p&gt;To associated the expression files
with the deployment
I needed to issue the command (once)&lt;/p&gt;
&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; nixops create ./webserver.nix ./ec2.nix &lt;span class=&quot;nt&quot;&gt;-d&lt;/span&gt; webserver
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;That and the actual deploy command
are the only two command line examples in this article.
That fact just now struck me as
unique and interesting:
most tutorials about
setting up servers involve a lot of
“copy and paste this into your shell.”&lt;/p&gt;

&lt;p&gt;(I’m fibbing a little:
I did need to
record AWS credentials
for NixOps to use.)&lt;/p&gt;

&lt;p&gt;Conversely, Nix supports and encourages
a descriptive approach.
You don’t make a series of
state dependent changes to the state of a server.
You describe what you want that server to look like
and tell Nix “make it like that.”&lt;/p&gt;

&lt;p&gt;That philosophy is hardly new -
Chef and Puppet both popularized
a decade ago.
Before that, CFEngine and tools like it
strove to make it possible.
Nix unifies the whole approach,
however,
establishing descriptive builds of everything
from individual software packages,
up through full network deployments.&lt;/p&gt;

&lt;p&gt;And the other command,
to examine the state of the servers
as I’ve amended it
(in this case, by adding a new post)
and realize my intent
on actual servers
so you can see it
looks like this:&lt;/p&gt;
&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; nixops deploy &lt;span class=&quot;nt&quot;&gt;-d&lt;/span&gt; webserver &lt;span class=&quot;c&quot;&gt;# &quot;make it like that&quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;
</description>
        <pubDate>Sat, 10 Feb 2018 00:00:00 +0000</pubDate>
        <link>http://judsonlester.info/2018/02/10/simple-webservers-with-nixops/</link>
        <guid isPermaLink="true">http://judsonlester.info/2018/02/10/simple-webservers-with-nixops/</guid>
        
        <category>technique</category>
        
        <category>ops</category>
        
        
      </item>
    
      <item>
        <title>A cryptographic toolchain</title>
        <description>&lt;p&gt;Strong authentication is often a requirement for working with software.
It can be a hassle to manage the secrets required to
adequately protect the integrity of our code
and control access to its deployment.
Many developers give up,
treating the whole game as not worth the candle.&lt;/p&gt;

&lt;p&gt;I’ve cobbled together a solution
involving a Yubikey secure element
that I like a lot,
both for its security,
and its convenience.
The whole process takes about 30 minutes.&lt;/p&gt;

&lt;h1 id=&quot;a-cryptographic-toolchain&quot;&gt;A Cryptographic Toolchain&lt;/h1&gt;

&lt;p&gt;What we’re going to do is set up a &lt;a href=&quot;https://www.yubico.com/product/yubikey-4-series/&quot;&gt;Yubikey&lt;/a&gt;
not only as a universal two-factor (U2F) token,
but also as a PGP smart-card.
The result will be a strong security element
that you can use to
sign into Github, Gmail and Dropbox,
sign and encrypt files
(including code commits and archives)
and authenticate to SSH servers (include Github for pushes.)
An added value is that the Yubikey can fit in your wallet,
which means you can carry your security keys
on your person,
much like physical keys.&lt;/p&gt;

&lt;h2 id=&quot;setting-up-the-yubikey&quot;&gt;Setting up the Yubikey&lt;/h2&gt;

&lt;p&gt;Install ykman (e.g. &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;brew install ykman&lt;/code&gt;)&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;it’s a python package, so it’s maybe possible to get through &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;pip&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Plug in your Yubikey.&lt;/p&gt;

&lt;p&gt;First thing to do:&lt;/p&gt;
&lt;div class=&quot;language-shell highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;ykman info
Device name: YubiKey 4
Serial number: 6646987
Firmware version: 4.3.7
Enabled connection&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;s&lt;span class=&quot;o&quot;&gt;)&lt;/span&gt;: OTP+U2F+CCID

Device capabilities:
    OTP:        Enabled
    U2F:        Enabled
    CCID:       Enabled
    OPGP:       Enabled
    PIV:        Enabled
    OATH:       Enabled
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The really important thing here is the firmware version.
Yubikeys with a firmware below 4.3.5
will need to be &lt;a href=&quot;https://www.yubico.com/keycheck/verify_otp&quot;&gt;replaced&lt;/a&gt;(it’s free!)&lt;/p&gt;

&lt;p&gt;We’d like to use the key for all of its functionality.
So let’s make sure everything is turned on.&lt;/p&gt;
&lt;div class=&quot;language-shell highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; ykman mode otp+u2f+ccid
Mode is already OTP+U2F+CCID, nothing to &lt;span class=&quot;k&quot;&gt;do&lt;/span&gt;...
&lt;span class=&quot;c&quot;&gt;# (you might be asked to confirm the change, and enter the admin password)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Next, we’d like to require the key to
wait for us to touch it before signing anything.
This prevents e.g. malware on our laptop from
using a plugged in key to sign bad things as if it were us.&lt;/p&gt;
&lt;div class=&quot;language-shell highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; ykman openpgp &lt;span class=&quot;nb&quot;&gt;touch &lt;/span&gt;aut on
Current &lt;span class=&quot;nb&quot;&gt;touch &lt;/span&gt;policy of AUTHENTICATE key is OFF.
Set &lt;span class=&quot;nb&quot;&gt;touch &lt;/span&gt;policy of AUTHENTICATE key to ON? &lt;span class=&quot;o&quot;&gt;[&lt;/span&gt;y/N]: y
Enter admin PIN:
Touch policy successfully set.

&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; ykman openpgp &lt;span class=&quot;nb&quot;&gt;touch &lt;/span&gt;sig on
Current &lt;span class=&quot;nb&quot;&gt;touch &lt;/span&gt;policy of SIGN key is OFF.
Set &lt;span class=&quot;nb&quot;&gt;touch &lt;/span&gt;policy of SIGN key to ON? &lt;span class=&quot;o&quot;&gt;[&lt;/span&gt;y/N]: y
Enter admin PIN:
Touch policy successfully set.
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Touch for encryption is probably unnecessary,
but it can’t hurt.
&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;ykman openpgp touch enc on&lt;/code&gt;&lt;/p&gt;

&lt;h2 id=&quot;u2f&quot;&gt;U2F&lt;/h2&gt;

&lt;p&gt;This is too easy &lt;em&gt;not&lt;/em&gt; to do.&lt;/p&gt;

&lt;p&gt;Go to sites that use U2F
(Google, Github)
and in your account settings,
look for Two-factor authentication devices.
Plug in the Yubikey,
choose “add authentication device”
and when prompted,
touch the blinking light.&lt;/p&gt;

&lt;h2 id=&quot;smart-card-setup&quot;&gt;Smart Card Setup&lt;/h2&gt;

&lt;p&gt;Now we’re going to set up a PGP keypair in the Yubikey.
We’ll use this to sign code,
but also to authenticate our pushes to Github,
and even sign in to SSH servers.&lt;/p&gt;

&lt;p&gt;Start with:&lt;/p&gt;
&lt;div class=&quot;language-shell highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;gpg2 &lt;span class=&quot;nt&quot;&gt;--card-edit&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This’ll put you into an interactive mode with GPG -
note the different prompt.&lt;/p&gt;

&lt;div class=&quot;language-shell highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;gpg/card&amp;gt; admin
gpg/card&amp;gt; passwd
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Enter &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;3&lt;/code&gt; to change the admin password.
You’ll be challenged with the &lt;em&gt;default&lt;/em&gt; admin password,
which is &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;12345678&lt;/code&gt;.
(At this phase, it won’t make it clear that it’s the default password,
although other GPG commands will remind you.)
Then for the new password,
and a confirmation repeat.
It’s my understanding that you want to limit yourself
to an 8-digit numeric code -
some environments will allow alphanumerics,
but others won’t, and you won’t be able to use
the Yubikey there.&lt;/p&gt;

&lt;p&gt;Then enter &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;1&lt;/code&gt; to change the user password.
Like the admin password,
you’ll need the &lt;em&gt;default&lt;/em&gt; password,
which is &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;123456&lt;/code&gt;.
The same advice about using
a 6-digit numeric code applies.&lt;/p&gt;

&lt;p&gt;Now, we want to generate a keypair on the key.
In my opinion,
this is one of the best features of this approach.
You generate a keypair in such a way that the private key
(the secret part)
never even leaves the smart-card -
which is designed to never let that data off of the device.&lt;/p&gt;
&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;gpg/card&amp;gt; generate
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;You’ll be asked for the keysize of three different keys
(signature, encryption and authentication)
which I set to &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;4096&lt;/code&gt; out of an abundance of caution.
On modern hardware
there’s pretty much no downside.
Note that the NEO can’t handle keys that large,
so stick to the still-decent &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;2048&lt;/code&gt; bit keys.&lt;/p&gt;

&lt;p&gt;You may also want to specify
personal details on the card itself -
these don’t carry over from the identifiers embedded in the keys.&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;gpg/card&amp;gt; list
...
gpg/card&amp;gt; name
...
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Finally,&lt;/p&gt;
&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;gpg/card&amp;gt; quit
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;You can check that gpg has a hold of the key by&lt;/p&gt;
&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; gpg2 --list-keys --keyid-format 0xlong
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;The new key should be the last one.&lt;/p&gt;

&lt;p&gt;Looks like&lt;/p&gt;
&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;...
pub   rsa4096/0x0123456789ABCDEF 2017-10-25 [SC]
      9EB2E24B3D545F30732E62A112E21E4B9A3F82AA
      0123456789012345678901234567890123456789
uid                   [ultimate] Judson Lester (General purpose key) &amp;lt;me@judsonlester.info&amp;gt;
sub   rsa4096/0x456789ABCDEF0123 2017-10-25 [A]
sub   rsa4096/0x789ABCDEF0123456 2017-10-25 [E]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Note that there are three keypairs above:
a master, signing key (the one marked &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;[SC]&lt;/code&gt;),
an authentication key (marked &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;[A]&lt;/code&gt;),
and an encryption key - &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;[E]&lt;/code&gt;.
There’s good reasons for this, but we mostly just need to know which is which.
Further on I’ll refer to some key ids like &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;$AUTHKEY_ID&lt;/code&gt;
which in the above is &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;0x456789ABCDEF0123&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Now we’re going to publish this key so that other people can use it.&lt;/p&gt;
&lt;div class=&quot;language-shell highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; gpg2 &lt;span class=&quot;nt&quot;&gt;--send-keys&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Next, we’ll pull the public information back onto the Yubikey from the keyservers.&lt;/p&gt;
&lt;div class=&quot;language-shell highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; gpg2 &lt;span class=&quot;nt&quot;&gt;--card-edit&lt;/span&gt;
gpg/card&amp;gt; fetch
gpg/card&amp;gt; quit
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;ssh-pubkey&quot;&gt;SSH pubkey&lt;/h2&gt;

&lt;p&gt;Even though GPG and SSH use the same kinds of basic cryptography,
SSH needs a different format for its public keys.
GPG has a handy tool to convert its public keys so that
SSH can use them:&lt;/p&gt;
&lt;div class=&quot;language-shell highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;gpg2 &lt;span class=&quot;nt&quot;&gt;--export-ssh-key&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;$AUTHKEY_ID&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; ~/.ssh/yubi-ssh.pub
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;Next, we’re going to set up gpg-agent so that it can act as an SSH agent.
It’s run on its own and provide your Yubikey keypair to
SSH for authentication
and to GPG for signing.&lt;/p&gt;

&lt;p&gt;Note: you have to spell our your home directory in the below -
replace $HOME with whatever &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;echo $HOME&lt;/code&gt; outputs.
&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;~/.gnupg/gpg-agent.conf&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;enable-ssh-support
write-env-file /home/$USERNAME/.gpg-agent-info
default-cache-ttl 28800
max-cache-ttl 86400
default-cache-ttl-ssh 28800
max-cache-ttl-ssh 86400
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;To make this work, you’ll need to add this to your &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;~/.bash_profile&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;export GPG_TTY=$(tty)

unset SSH_AGENT_PID
if [ &quot;${gnupg_SSH_AUTH_SOCK_by:-0}&quot; -ne $$ ]; then
  export SSH_AUTH_SOCK=&quot;$(gpgconf --list-dirs agent-ssh-socket)&quot;
fi
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;That way SSH and GPG will know where to find the GPG agent.
Once you’ve done that, to get the configs picked up in
any shell that was already running
(like the current one):&lt;/p&gt;
&lt;div class=&quot;language-shell highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; &lt;span class=&quot;nb&quot;&gt;source&lt;/span&gt; ~/.bash_profile
&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; gpgconf &lt;span class=&quot;nt&quot;&gt;--kill&lt;/span&gt; gpg-agent
&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; gpg-connect-agent &lt;span class=&quot;se&quot;&gt;\b&lt;/span&gt;ye
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h3 id=&quot;other-keys-in-agent&quot;&gt;Other keys in agent&lt;/h3&gt;

&lt;p&gt;You very likely have other SSH keys you’d like to use.
You can use &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;ssh-add&lt;/code&gt; to put other keys into the gpg-agent.
You’ll be challenged for their passphrase once on the command line
and then &lt;strong&gt;again&lt;/strong&gt; for a new passphrase for their use in gpg-agent.
Honestly, I tend to use the same passphrase for simplicity.&lt;/p&gt;

&lt;p&gt;From then on, if SSH needs that key,
gpg-agent will use its pinentry to ask for
the “GPG” passphrase
(i.e. the new one you enter as you ssh-add).&lt;/p&gt;

&lt;p&gt;From now on, when SSH needs that key,
GPG agent will challenge you for its passphrase,
and then make it available to SSH for a set period.&lt;/p&gt;

&lt;h2 id=&quot;git-signing&quot;&gt;Git signing&lt;/h2&gt;

&lt;p&gt;Next, we want to configure Git to use our GPG keypair.&lt;/p&gt;

&lt;p&gt;Add this to &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;~/.config/git/config&lt;/code&gt; (or &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;~/.gitconfig&lt;/code&gt;)&lt;/p&gt;
&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;[user]
  signingkey = {last 6 hex of signing key id (e.g. ABCDEF)}

[commit]
  gpgSign = true
[tag]
  forceSignAnnotated = true
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now when you commit,
or do annotated tags like:&lt;/p&gt;
&lt;div class=&quot;language-shell highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;git tag &lt;span class=&quot;nt&quot;&gt;-m&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;Annotated, therefore signed&quot;&lt;/span&gt; a-tag
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;git will use your signing key on the item.&lt;/p&gt;

&lt;h2 id=&quot;github-settings&quot;&gt;Github settings&lt;/h2&gt;

&lt;p&gt;Copy &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;~/.ssh/yubi-ssh.pub&lt;/code&gt; to the SSH pubkeys.
This will let you use your Yubikey to authenticate pushes.&lt;/p&gt;

&lt;p&gt;If you need to specify a key in your &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;~/.ssh/config&lt;/code&gt;,
use &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;~/.ssh/yubi-ssh.pub&lt;/code&gt;.
This confused me,
since you usually identify keypairs to SSH
by the private key file,
but that “file” is intentionally unavailable.&lt;/p&gt;

&lt;p&gt;Copy the output of &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;gpg2 --export --armor ${SIGNING_ID}&lt;/code&gt; to GPG keys.
This will get you the good feeling of “verified” badges next to your commits.&lt;/p&gt;

&lt;h2 id=&quot;notification&quot;&gt;Notification&lt;/h2&gt;

&lt;p&gt;Everything up until now has been simple configuration of stock tools.
The one irritation I had was that,
while I wanted the Yubikey to require interaction before signing anything
(because otherwise any process on the computer could get access to the keys),
the quietly blinking “Y” wasn’t quite enough
to get my attention all the time.
What happens is that I’d make a &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;git commit&lt;/code&gt; and
look away, task complete,
only to have the signature part time out and fail.&lt;/p&gt;

&lt;p&gt;So,
I built a little notification engine for the Yubikey.&lt;/p&gt;

&lt;p&gt;For now, basically, see
https://github.com/nyarly/socket-notify&lt;/p&gt;

&lt;p&gt;You’ll need Cargo
(i.e. the Rust dev environment)
set up on your machine to make it work,
but that’s easier than you might expect.&lt;/p&gt;

&lt;p&gt;First, run&lt;/p&gt;
&lt;div class=&quot;language-shell highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;cargo &lt;span class=&quot;nb&quot;&gt;install &lt;/span&gt;socket-notify
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Then, put this into &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;~/.gnupg/scdaemon.conf&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;log-file socket:///tmp/scdaemon.sock
debug 1027
debug-assuan-log-cats 511
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Then run&lt;/p&gt;
&lt;div class=&quot;language-shell highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;gpg-connect-agent scd killscd
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;That should restart the program that interacts with the Yubikey as a smartcard.&lt;/p&gt;

&lt;p&gt;Make sure socket-notify runs when your computer is running
(I use systemd user services for this, but it depends on your OS.)&lt;/p&gt;

&lt;p&gt;The upshot is that when a program challenges the Yubikey for a signature,
you’ll get a desktop notification to remind you that you need to touch the button.&lt;/p&gt;

&lt;h2 id=&quot;revocation-certificate&quot;&gt;Revocation certificate&lt;/h2&gt;

&lt;p&gt;If you lose your yubikey,
you want to publish the fact that you can’t use that key anymore.
You want to prove that you,
the owner of the key,
are the one publishing that the key isn’t any good.
So obviously you want to sign a message to that effect.
But you won’t have the key to sign it with anymore.
The way out of this paradox is to generate (and protect)
“revocation certificates,”
essentially pre-signed claims that the key is bad,
ready to distribute when the worst happens.&lt;/p&gt;

&lt;p&gt;Current versions of &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;gpg2&lt;/code&gt; will automatically generate a revocation certificate.
If you don’t see a message about “revocation certificate stored as …”,
(or even if you aren’t sure)
you can always&lt;/p&gt;
&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;gpg2 --gen-revoke 0x0123456789ABCDEF &amp;gt; ~/.gnupg/openpgp-revocs.d/yubikey-ABCDEF.gpg.asc
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;You can also generate multiple certificates -
one for each computer you use your Yubikey on, for instance.&lt;/p&gt;

&lt;p&gt;You’ll want to protect that file carefully -
some suggest backing it up to removable media.
It can’t be used to impersonate you,
but if it’s published to a keyserver,
other users of GPG will have a hard time trusting the key again.
Which is the point.&lt;/p&gt;

&lt;h3 id=&quot;actually-revoking-the-key&quot;&gt;Actually revoking the key&lt;/h3&gt;

&lt;p&gt;If you lose your Yubikey,
(or, say, it turns out to be theoretically vulnerable to ROCA…)
you revoke the key like this:&lt;/p&gt;
&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;gpg2 --import ~/.gnupg/openpgp-revocs.d/yubikey-ABCDEF.gpg.asc
gpg2 --send-keys
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;which adds the certificate to your local keyring,
and then publishes it to the keyservers.&lt;/p&gt;

&lt;p&gt;Then, start over with this guide.&lt;/p&gt;
</description>
        <pubDate>Fri, 03 Nov 2017 00:00:00 +0000</pubDate>
        <link>http://judsonlester.info/2017/11/03/a-cryptographic-toolchain/</link>
        <guid isPermaLink="true">http://judsonlester.info/2017/11/03/a-cryptographic-toolchain/</guid>
        
        <category>technique</category>
        
        <category>practice</category>
        
        
      </item>
    
      <item>
        <title>Semantic Versioning, in Reverse</title>
        <description>&lt;p&gt;As phenomenal a concept as it is,
semantic versioning
often gets a bad rap.
Many developers
publicly express their frustration
(or even contempt)
for this tidy codification of
a core principle of software design:
the definition of interfaces.
I tend to think that it’s not
semantic versioning
that’s badly understood,
but the idea of
what an interface is
and how to design one.&lt;/p&gt;

&lt;h2 id=&quot;a-quick-recap&quot;&gt;A Quick Recap&lt;/h2&gt;

&lt;p&gt;&lt;a href=&quot;http://semver.org&quot;&gt;Semantic versioning&lt;/a&gt;
(sometimes called “semver”)
is a relatively important concept in modern software development.
It’s a formulation of rough understandings of
what the various parts of a version number mean
in terms that are generally agreed upon.&lt;/p&gt;

&lt;p&gt;In semver,
a software version
(in general this is loosely true, but in semver it is strictly so)
has three parts:
The major version number,
the minor version number,
and a subversion number (sometimes called the patchlevel.)
The three parts are connected with periods, like this:
&lt;strong&gt;1.3.5&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Semver specifies that those three parts have very distinct meanings.
The major version changes when there are breaking changes to the interface.
The minor version changes when new features are added to the software.
The subversion changes for any other release of the software.&lt;/p&gt;

&lt;h2 id=&quot;motivation&quot;&gt;Motivation&lt;/h2&gt;
&lt;p&gt;The value of semantic versioning as a practice comes from
the terse, precise communication about
the compatibility of software over time.&lt;/p&gt;

&lt;p&gt;Faced with a software problem,
we often imagine, correctly,
that someone has had a similar problem and solved it already.
When we find such a library,
now the problem becomes making good use of that library.&lt;/p&gt;

&lt;p&gt;It’s extremely valuable to be able to
find code ready-made to provide functionality we require,
especially when it involves the implementation of a standard protocol
or the encapsulation of an API.
Truthfully, there are many reasons that a code library might be
useful,
and why we’d want to make it reusable.&lt;/p&gt;

&lt;p&gt;Like everything in engineering,
there are trade-offs.
And one of the trade-offs of reusable software
is maintaining compatibility.
It’s frustrating when
a library your software has come to depend on
releases and update,
and how your code won’t compile because
it expects features of that library that have changed.
How many hours (or years!) of developer time
have been burnt in the effort
to adapt to a shifting interface?&lt;/p&gt;

&lt;p&gt;One longstanding solution to using that code is to
copy it wholesale into your code and
use it as if you’d written it from whole cloth.
Some modern systems persist with this approach,
often describing the procedure as
“checking in vendored code”
or simply “vendoring.”
The chief and abiding advantage to this approach
is that you don’t have to worry about
the boundary between your code and
the third-party library anymore.
It’s all yours, for good or ill.&lt;/p&gt;

&lt;p&gt;If there were bugs in the library at the moment we copied it,
now those are our bugs to maintain.
Since we’re not the original author,
we don’t have the advantage of bug reports from other users.
So, those bugs are likely to be fixed by the upstream maintainer.
More urgently, fixes to security issues
should be propagated as quickly as possible,
but we’re sitting on an old copy of the code.&lt;/p&gt;

&lt;p&gt;Beyond the very real maintenance issues,
it’s easy (but often wrong) to
disregard new features made available in newer versions.
Often those new features are attractive for
the same reason that the library was attractive in the first place:
they’ll save us implementation time.&lt;/p&gt;

&lt;p&gt;The problem is, how can we know that
updating to a new version of the software
won’t break the application that depends on it?
Maybe we’ll need to do more work
keeping up with changes in the library we’re tracking
than we would to simply do just the maintenance we need ourselves.
Of course, we can’t know for certain
whether that’s true.
Once you start considering the possibility
you’ll always wonder about
what could have been.&lt;/p&gt;

&lt;p&gt;So we’re caught in a bind:
we can choose to never update
and accept or remediate bugs and security flaws,
or we can adopt code wholesale and become a second maintainer,
or we can cope with the breakages that occur when we update.
All of these choices sounds like terrible propositions.
The virtue of semantic versioning is that we can
reduce the cost of the last option,
by making the breakages more predictable,
or even offering the assurance that a particular version
won’t break our application.&lt;/p&gt;

&lt;p&gt;So, as a consumer of third-party libraries,
the value of well managed semver is pretty clear.
It’s a way out of the update bind.
As a maintainer,
we get the good feeling that we’re “taking care” of our users.
More concretely, we can hope and expect that
our software will see greater adoption,
which we might desire for a number of reasons.&lt;/p&gt;

&lt;p&gt;Most of all, semantic versioning is about
recognizing that writing software is a human act:
we write software for other humans.
Part of doing things with and for each other
is communicating about what we’re doing.
Semantic versioning provide really powerful support for that communication.
It’s a well known shorthand
for complicated ideas,
as well as being a social prompt to
talk about the context your software expects.&lt;/p&gt;

&lt;p&gt;But also don’t ignore the
case where the maintainer of a library is also its primary consumer.
There’s amazing value in being able to set down one project
sometimes for months,
and work on another than consumes its interface
without worrying about the details inside it.
Clear communication with yourself
months from now
is at least as valuable as
communication with a larger community of developers.&lt;/p&gt;

&lt;h2 id=&quot;in-practice&quot;&gt;In Practice&lt;/h2&gt;

&lt;p&gt;Every new release of a piece of software
should have a new version number.
The versions should always increase.
In some ways, the version acts as
a kind of counter,
marking out releases on a timeline.&lt;/p&gt;

&lt;p&gt;But every new release &lt;em&gt;could&lt;/em&gt;
increase any of the three components
of the version.
So how do we figure out
what version our release should be?&lt;/p&gt;

&lt;p&gt;The first release is easy: start with 0.0.1
and no one can say that you’re wrong.
But after that, how do we decide how to
increment a version?
In other words, should we bump
the major,
minor,
or subversion
for this release?&lt;/p&gt;

&lt;p&gt;First of all, the best part about semver it that
until the 1.0 release,
all bets are off.
We’re free to make releases and use them in our own code,
or let others try it out and make suggestions,
without yet committing to anything about the interface.
That’s a huge and important freedom,
and one to make great use of.&lt;/p&gt;

&lt;p&gt;By the same token,
0.x releases make no promises.
Anything can change at any time,
which is good in the early days
of a new package.
But when you’re trying to &lt;em&gt;use&lt;/em&gt; the package,
that’s really a drag.&lt;/p&gt;

&lt;p&gt;At some point we decide
“This!
This is how this should work,
and how it should be used!”
That decision point gets codified in documentation.
&lt;em&gt;That’s&lt;/em&gt; when we release the 1.0:
as a way of announcing the stabilization of the interface.&lt;/p&gt;

&lt;p&gt;The 1.0 release is important.
It’s a moment when we say to all our consumers
(who might be ourselves, five minutes in the future)
“Trust me to shepherd this code.”
We’re making a commitment,
and we should take that seriously.&lt;/p&gt;

&lt;p&gt;From then on,
whenever we make a new release,
we need to consider what kind of change we’ve made:
breaking changes update the major version,
new features update the minor version,
bug fixes update the patchlevel.
Easy!&lt;/p&gt;

&lt;p&gt;Of course,
that reasoning presupposes that we understand
what makes for a breaking change.
And to reason about that,
we have to have decided on our package’s
&lt;em&gt;interface.&lt;/em&gt;&lt;/p&gt;

&lt;h3 id=&quot;an-aside-about-private&quot;&gt;An Aside about &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;private&lt;/code&gt;&lt;/h3&gt;

&lt;p&gt;That’s the issue I really want to address, though.
I come to think that some of the furor against semver is because
we’ve all failed to understand and disseminate
the art of design and maintenance of a software interface.&lt;/p&gt;

&lt;p&gt;As a for instance, see if your experience gibes with mine:
when I was first learning object oriented programming,
(in Java)
I was really confused by the access keywords.
I think everyone plays with trivia about
what the difference is between &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;public&lt;/code&gt; and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;protected&lt;/code&gt; is.
Like, in Java,
&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;protected&lt;/code&gt; is defined in relationship to &lt;em&gt;packages&lt;/em&gt;,
but &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;private&lt;/code&gt; is in relationship to classes,
or something like that.
but in Ruby
&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;protected&lt;/code&gt; means “defined by a superclass”
&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;private&lt;/code&gt; means “only with an implicit receiver”.
What the access keywords &lt;em&gt;mean&lt;/em&gt;
is one question.&lt;/p&gt;

&lt;p&gt;But the really pointed question is: what are they &lt;em&gt;for?&lt;/em&gt;
I harbored this understanding that
they were somehow defensive -
that &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;private&lt;/code&gt; methods were somehow “more secure”
than &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;public&lt;/code&gt; ones.
In retrospect,
the security idea is laughable,
but I’m pretty sure I’m not alone.
But &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;private&lt;/code&gt; is safer than &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;public&lt;/code&gt;
in a very different sense.&lt;/p&gt;

&lt;p&gt;Later, relying on Rubydoc to browse code,
I was frequently frustrated by other people using &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;private&lt;/code&gt;,
since it implicitly hid the code from documentation.&lt;/p&gt;

&lt;p&gt;If you’re paying careful attention,
you’ll notice that
I’m talking about using a &lt;em&gt;documentation&lt;/em&gt; tool
to &lt;em&gt;browse code.&lt;/em&gt;
Because of the tragic state of documentation
(and Github README files are not helping…)
that’s really what those tools are good for anymore.
Another way to put that is
I went to use Rubydoc to find out what a particular method does,
and how it interprets its parameters,
and the docs were either &lt;em&gt;not there&lt;/em&gt; or &lt;em&gt;not useful.&lt;/em&gt;
But there was a “show source” button, and I could puzzle it out from there…
until there are method calls in the source body.
No problem, I can just find the docs for those methods and repeat…
unless those methods are marked &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;private&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Given the wretched state of documentation in general,
(and for a prime example, check out &lt;a href=&quot;https://rubygems.org/profiles/nyarly&quot; title=&quot;i.e. mine&quot;&gt;this yokel’s profile.&lt;/a&gt;)
I did two things: I started using &lt;a href=&quot;http://yardoc.org/&quot;&gt;YARD&lt;/a&gt;
(because it can be configured to ignore access keywords for the purpose of documentation)
and I stopped adding access keywords to my own code.
Everything was &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;public&lt;/code&gt;, and I liked it that way.&lt;/p&gt;

&lt;p&gt;What became clear though, is that
I was implicitly exposing every method in my libraries as their interface.
When I tried to update libraries I’d released
I’d receive a bunch of issue reports that the new version didn’t work
with existing client code.
At one point I considered never changing any of the methods in code I’d released,
but the maintenance nightmare there was too daunting.
The point was,
this is &lt;em&gt;exactly&lt;/em&gt; what marking methods &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;private&lt;/code&gt; is all about:
those are the methods we’re saying are off limits
to outside consumers of our code,
and it’s off limits so that we’re free to change it.
Seeing how access keywords were being used poorly,
I’d foolishly abandoned a useful and powerful tool.&lt;/p&gt;

&lt;p&gt;The truth is that access keywords are
a kind of enforced documentation.
They let us establish for other developers
(first, and most of all)
what parts of our library are its interface,
and what’s retained as internal implementation.
Best of all, the rules about use are enforced
at the language level, so
there cannot be any confusion about what’s what:
if the compiler allows it, it’s in the public interface.&lt;/p&gt;

&lt;p&gt;Here we find another trade off.
Private aspects of a library are easier to change over time
because only the library itself uses them,
and if its tests all pass, we can be confident we haven’t broken anything.
But to be useful, we have to
expose some parts as public.
What to expose means predicting which uses will be
most common
or most powerful
for the consumers of the library.
Exposing everything means we leave that decision up to our users,
but it means that every change will break somebody’s program.&lt;/p&gt;

&lt;p&gt;The simple rule about marking code entities
(that is: functions, methods, classes, constants, whatever)
public is one way you’re saying to the consumers of your code
“this is the interface - this is how to use it.”
When I was marking everything as public,
I was mixing in the useful interfaces with
little utility functions.
If you’re using my library,
you’re within your rights to make use of anything marked &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;public&lt;/code&gt;.
And if I remove those methods or change their signatures,
you’re right to be (loudly) annoyed;
the responsible thing, in fact, is to cease to use my library.
After all, it’s causing unplanned maintenance work to keep up with,
and it’s likely to keep causing that work.&lt;/p&gt;

&lt;h1 id=&quot;from-access-to-interface&quot;&gt;From Access to Interface&lt;/h1&gt;

&lt;p&gt;Moving quickly on from issues of open-source entitlement and programmer motivation,
(which are both important and well covered elsewhere)
imagine the following case:
if all my methods are public,
and the audience for my excitingly popular library use all of them
what do I do when I want to change one of those methods?&lt;/p&gt;

&lt;p&gt;I might want
to provide a new feature,
or reduce repetition,
or improve the structure of the code.
There’s tons of reasons to make changes to a library,
but if all my methods are public and therefore consumed
(even just theoretically)
by some client code
the I can’t get any of the value of making those changes
without breaking someone else’s program.&lt;/p&gt;

&lt;p&gt;The interface also serves as
a point of entry,
an introduction to a component for newcomers.
An idiomatic interface communicates
the purpose
and abilities of a module.
A small interface
focuses that communication to the things that are important,
where a broad interface risks losing that focus.
There are times where that’s acceptable,
but exposing everything as public discards that value
without good purpose.&lt;/p&gt;

&lt;p&gt;Since interfaces serve as the seams
between pieces of an application,
they also serve as natural points of instrumentation.
The interfaces are where interaction happens,
and we can meaningfully inspect that interaction
to get a view on what the application is doing.&lt;/p&gt;

&lt;p&gt;Likewise,
interfaces are excellent subjects of testing,
both as boundaries of a tested unit,
and also as the template for dummies
to replace the component when testing.&lt;/p&gt;

&lt;p&gt;Like any useful communication,
designing interfaces can be hard.
Specifically, deciding where the interface is,
and what’s not can be hard.
Why else would well meaning designers cede the question entirely?
The actual process is pretty open,
and I’ve had good experiences with many different approaches.
Sometimes, I know well enough what the code needs to do
that I can define the interface up front,
write tests first
and let the tests drive the development.
Other times, libraries arise from an existing application,
and the trick is to figure out
where the boundary between the specific application
and the reusable library lies.
A lot of the time, though,
I discover an interface only as the software comes into being,
and gets real-world use.
Regardless, the problem to solve is where that boundary is
and what the points of access are through the boundary.&lt;/p&gt;

&lt;p&gt;Incidentally, automated tests are great for this.
Even if you don’t do test driven &lt;em&gt;development&lt;/em&gt;,
using tests to drive the &lt;em&gt;design&lt;/em&gt;
will lead to nicer interfaces,
and the tests will help control regressions between versions of your library.
Plus, well written tests can go a long way towards documenting the interface.
It must be said though,
they don’t replace the need for good prose documentation.&lt;/p&gt;

&lt;h2 id=&quot;when-last-we-saw-our-heroes&quot;&gt;When Last We Saw Our Heroes&lt;/h2&gt;

&lt;p&gt;But we were talking about how to apply semver to our projects.&lt;/p&gt;

&lt;p&gt;With a solid interface, we can set the 1.0 version.
From there, the process of determining the version for a new release is straightforward.
We simply ask ourselves
the questions implied by the semver definition
and go from there.
That is, first we look at
this release
as compared to the last
to find what changed.&lt;/p&gt;

&lt;p&gt;Then we ask:
“did we change the interface?
Are there methods or types missing,
or still present but changed in incompatible ways?
Did any methods change their parameters?”
If the answer is yes, then we need to bump the major version.&lt;/p&gt;

&lt;p&gt;“Did we add to the interface?
Are there new methods or types?
Did existing methods learn to accept a wider array of values?”
If the answer is yes, we need to bump the minor version.&lt;/p&gt;

&lt;p&gt;“Are all the changes in this revision to fix bugs?
Are the changes or additions from the previous two questions
&lt;strong&gt;only&lt;/strong&gt; made because we’d released an implementation that
doesn’t support the interface we’d intended?”
If we can answer this yes, then we can get away with a subversion bump.&lt;/p&gt;

&lt;p&gt;So, the hard part is setting the interface.
Keeping semver is actually pretty easy.
Right?&lt;/p&gt;

&lt;h2 id=&quot;leader-and-follower&quot;&gt;Leader and Follower&lt;/h2&gt;

&lt;p&gt;A careful read will notice there’s some tension in those questions.
We’re letting some major version bumps be reduced to subversions
in certain circumstances.
How is that okay?&lt;/p&gt;

&lt;p&gt;In a perfect world,
it wouldn’t be.
We’d never need to release subversion
bumps in the first place.
Heck, we wouldn’t need versions in the first place,
because all software would be perfect on the first release.
But the fact is, we make mistakes.
There’s a point of view that says that the release
is the ultimate arbiter of the change,
and that just because you didn’t intend a particular interface,
that’s what you released,
so you’re stuck making a major version release.&lt;/p&gt;

&lt;p&gt;My personal opinion is that it’s more nuanced than that.
Hard lines on how versions get set lead to us supporting
bug-level compatibility in our software forever.
And one of the virtues of making semver releases,
(and of releasing open source software)
is being able to say
“if you needed this bug,
lock to that exact version.”&lt;/p&gt;

&lt;p&gt;That said,
it’s certainly better to avoid
“interface not as intended” subversion releases.
Especially if there’s been a lot of adoption of the previous version
(and a lot of package distribution systems will let you count downloads)
or if a lot of time has passed
(which is essentially a proxy for adoption.)&lt;/p&gt;

&lt;p&gt;Even more so, we’d like to avoid making major version releases.
If they happen with real frequency,
we’re not really providing benefit to anyone by using semver.
The whole point of major versions
is that they signal that we’ve broken the interface.
If we do that frequently,
then we give up one of the key advantages of semver.&lt;/p&gt;

&lt;p&gt;Likewise, we’d like to be careful about our minor version releases.
Each implies an extension of the interface,
which starts to erode the focus of purpose
that our interface should communicate.
It ends up making it harder for newcomers to understand
what our code does and how to use it.
There’s a way in which each minor version release implies
we didn’t know what the interface should look like,
and so maybe we still don’t know what it should look like.
(There’s a Halting Problem analogue there,
but it’s a variety that’s hardly unique in our discipline.)&lt;/p&gt;

&lt;h2 id=&quot;at-last-the-point&quot;&gt;At Last, The Point&lt;/h2&gt;

&lt;p&gt;So here we come, finally,
to the core thesis.
If we’re going to try to avoid major and minor version releases,
we need to avoid making changes to our interface,
either breaking changes or additions.
And to avoid those changes, we need to consider the changes we’re &lt;em&gt;planning&lt;/em&gt; to make,
not just the changes we’ve actually made since the last release.&lt;/p&gt;

&lt;p&gt;In other words, rather than figuring out how our version is affected by our changes,
we need to figure out how our planned changes with affect our version.
That’s what I mean by “semantic versioning in reverse,”
and it’s actually part of the practice of
software engineering,
and part of why it might be reasonably called
a discipline.&lt;/p&gt;

&lt;h2 id=&quot;on-the-how&quot;&gt;On the How&lt;/h2&gt;

&lt;p&gt;If you’re convinced,
or just curious,
you might be asking
“okay, but how do I get there?”
First of all,
the structure of this essay is advised.
I really think that you learn to do
the “well considered changes” thing
by first making what changes seem called for,
and then categorizing them.
That’s part of why semantic versioning is so useful:
if only as an exercise towards considered change,
it’s amazingly valuable.&lt;/p&gt;

&lt;p&gt;As time goes on,
and on smaller projects,
the change to version
can become a useful shorthand for examining
contemplated changes.
And, indeed, towards discovering where your interfaces lie
and what they should look like.
One larger projects,
the same skills come to bear,
but it may require more formal processes to control well.&lt;/p&gt;

&lt;p&gt;As an example of a lightweight process,
check out how &lt;a href=&quot;http://emberjs.com/deprecations/&quot;&gt;Ember.js&lt;/a&gt; handles this.
They’ve set up in-library support for deprecating parts of their interface.
If you use a deprecated method, you get a warning,
and that’s all.
When a major version lands,
all the deprecated parts of the interface go away.
There’s drawbacks to this approach
(people complain about “deprecation spew”)
but it means that the upgrade path to the next major version
should be clean and straightforward.&lt;/p&gt;
</description>
        <pubDate>Sun, 30 Oct 2016 00:00:00 +0000</pubDate>
        <link>http://judsonlester.info/2016/10/30/semantic-versioning-in-reverse/</link>
        <guid isPermaLink="true">http://judsonlester.info/2016/10/30/semantic-versioning-in-reverse/</guid>
        
        <category>engineering</category>
        
        <category>practice</category>
        
        
      </item>
    
      <item>
        <title>Nginx + Passenger on Gentoo</title>
        <description>&lt;p&gt;From the “niche interest” desk:&lt;/p&gt;

&lt;p&gt;Nginx builds all of it’s components into one executable (they argue that it’s
necessary for their level of speed and security). Gentoo lets you configure
software all kinds of ways, so it’s pretty easy to configure exactly which of
the myriad of modules you want to install.  But like every Linux distro,
they’ve had an ongoing tete-a-tete with Rubygems, so Passenger isn’t supported
out of the box in the Gentoo set of options.  Passenger’s idea of installing
itself  into Nginx is to recompile the whole thing
- &lt;em&gt;without&lt;/em&gt; respecting the modules that were already installed.&lt;/p&gt;

&lt;p&gt;It’s a headache, and I’ve given up trying to assign blame - I think probably
everyone solved the (very large) problems in front of  them, and it happens
that their solutions don’t interact well. (Gentoo Rubyists (and yes, I’m sure
the plural there is optimistic): I’m liking chruby + direnv + bundler as an
adjunct to eselect.)&lt;/p&gt;

&lt;p&gt;But, there’s a couple of hidden outs that make it entirely feasible to get a
respectable nginx/passenger deploy going on Gentoo, which I’ll record here for
my own purposes, at least.&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-bash&quot; data-lang=&quot;bash&quot;&gt;gem &lt;span class=&quot;nb&quot;&gt;install &lt;/span&gt;passenger
&lt;span class=&quot;nb&quot;&gt;mkdir&lt;/span&gt; /etc/portage/env
&lt;span class=&quot;nb&quot;&gt;echo&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;NGINX_ADD_MODULES=&apos;&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;$(&lt;/span&gt;passenger-config &lt;span class=&quot;nt&quot;&gt;--nginx-addon-dir&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&apos;&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; /etc/portage/env/nginx-passenger
&lt;span class=&quot;nb&quot;&gt;echo&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;www-servers/nginx nginx-passenger&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;gt;&amp;gt;&lt;/span&gt; /etc/portage/package.env
emerge nginx&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;In /etc/nginx/nginx.conf, add:&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-bash&quot; data-lang=&quot;bash&quot;&gt;passenger_root /usr/local/lib64/ruby/gems/2.0.0/gems/passenger-4.0.29/&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
passenger_ruby /usr/bin/ruby20&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

</description>
        <pubDate>Fri, 15 Jan 2016 00:00:00 +0000</pubDate>
        <link>http://judsonlester.info/2016/01/15/nginx-passenger-on-gentoo/</link>
        <guid isPermaLink="true">http://judsonlester.info/2016/01/15/nginx-passenger-on-gentoo/</guid>
        
        <category>technique</category>
        
        <category>ops</category>
        
        
      </item>
    
      <item>
        <title>Three Corner Rsync</title>
        <description>&lt;p&gt;This is such a good trick, I wanted to share.&lt;/p&gt;

&lt;p&gt;Here’s the situation: you have a ton of files on Old Server, and you want to
get them to New Server. Both of them have nice fat pipes - they may even be on
the same 10G ethernet switch.&lt;/p&gt;

&lt;p&gt;You could do something like:&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-shell&quot; data-lang=&quot;shell&quot;&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; rsync &lt;span class=&quot;nt&quot;&gt;-av&lt;/span&gt; old-server:/files/ /tmp/old-server/
&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; rsync &lt;span class=&quot;nt&quot;&gt;-av&lt;/span&gt; /tmp/old-server/ new-server:/files&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;The trouble with this is that it’s sending files over the public internet,
using up your local network connection, and it’s sending the files over that
slower network &lt;em&gt;twice.&lt;/em&gt; Excuses to grab a cup of coffee aside, there has to be
a bette way.&lt;/p&gt;

&lt;p&gt;Or you could go onto one of the servers and set up an SSH account for the other
server and shuffle keys around and all that. But that’s a big hassle, and you
have to remember to get rid of the unnecessary account afterwards. Which if
you’re me, you’ll forget to do.&lt;/p&gt;

&lt;p&gt;But &lt;em&gt;you&lt;/em&gt; already have an account on both machines with you public key in an
authorized_keys file somewhere.&lt;/p&gt;

&lt;p&gt;Now, we make sure that we have the keys for the new-server in our agent:&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-shell&quot; data-lang=&quot;shell&quot;&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; ssh-add ~/.ssh/new_server_rsa
&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; ssh-add ~/.ssh/old_server_rsa&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;With the preliminaries out of the way, the real juice looks like this:&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-shell&quot; data-lang=&quot;shell&quot;&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; ssh &lt;span class=&quot;nt&quot;&gt;-A&lt;/span&gt; old-server &lt;span class=&quot;s1&quot;&gt;&apos;rsync -av /files/ new-server:/files/&apos;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;And now you’ve got a full-pipe transfer going from the old server to the new
one, using the credentials stored in your local agent to authenticate you first
from your console to the old-server, and then from the old-server to the new
one. You’ll get the usual rsync update output as the transfer goes, and at the
end everything will be transferred over.&lt;/p&gt;

&lt;p&gt;Frankly, I think there ought to be a merit badge or something. It’s that cool&lt;/p&gt;

&lt;h2 id=&quot;control-sockets&quot;&gt;Control Sockets&lt;/h2&gt;

&lt;p&gt;If you’ve set up master sockets for your servers (which is very handy), the
above might not work. The reason is that SSH set’s up agent forwarding (which
is what the &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;-A&lt;/code&gt; in the above does) when it opens the connection to a server.
If you’re using master sockets, that connection has already been established,
and SSH will (silently) ignore the request to forward your agent connection.&lt;/p&gt;

&lt;p&gt;In that case, start out with this:&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-shell&quot; data-lang=&quot;shell&quot;&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; ssh &lt;span class=&quot;nt&quot;&gt;-O&lt;/span&gt; &lt;span class=&quot;nb&quot;&gt;exit &lt;/span&gt;old-server
&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; ssh &lt;span class=&quot;nt&quot;&gt;-O&lt;/span&gt; check old-server&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;You want something like: &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;&quot;Control socket ... No such file or
directory&quot;&lt;/code&gt; Otherwise, it means that you have an existing connection to
old-server. Usually you have an existing SSH session open, and you can just
close it. If you’re feeling lazy and irresponsible, the message we don’t want
says something like &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;&quot;Master running (pid=12345)&quot;&lt;/code&gt; - you can just
kill the master SSH process and end the connection that way…&lt;/p&gt;

&lt;h2 id=&quot;a-proviso&quot;&gt;A Proviso&lt;/h2&gt;

&lt;p&gt;The one problem with this technique is that it relies on something called
‘agent forwarding,’ which can have security implications. If you’re worried
about that, setting up the temporary account between servers is probably your
best bet.&lt;/p&gt;
</description>
        <pubDate>Fri, 01 Jan 2016 00:00:00 +0000</pubDate>
        <link>http://judsonlester.info/2016/01/01/three-corner-rsync/</link>
        <guid isPermaLink="true">http://judsonlester.info/2016/01/01/three-corner-rsync/</guid>
        
        <category>technique</category>
        
        <category>ops</category>
        
        
      </item>
    
  </channel>
</rss>
