Programming on Miek Gieben

Forgejo Go Benchstat

Fri, 30 Jan 2026 17:53:04 +0100

For the longest time I wanted to see the performance impact of my changes. This desire goes back years, back to the early days of CoreDNS. And now I finally have it.

It does require running your own forgejo-runner on your own hardware to have any meaningful data. Without further adu:

# On https://codeberg.org/miekg/dns/actions you can see the run and copy and paste a new.txt if performance is
# better. Put this contents in old.txt
on: [push, pull_request]

jobs:
  test:
    runs-on: atoom-tiny
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-go@v6
        with:
          go-version: ${{ vars.GOVERSION }}
      - run: |
          apt-get update
          apt-get -y install dnsperf jq
          go install golang.org/x/perf/cmd/benchstat@latest
          (cd cmd/reflect; go build)
          HOSTNAME=atoom-tiny
          # Sleep here so we have the machine to ourselves.
          sleep 60
          (cd cmd/reflect; make new.txt)
          REF=$(echo ${{ forgejo.ref }} | cut -d/ -f3)
          echo ${{ forgejo.api_url}}/repos/${{ forgejo.repository }}/issues/${REF}/comments
          echo "{\"body\": $(cd cmd/reflect; make stat | jq -sR .)}" | \
                          curl --json @- \
                          -H 'Authorization: token ${{ forgejo.token }}' \
                          ${{ forgejo.api_url}}/repos/${{ forgejo.repository }}/issues/${REF}/comments

See https://codeberg.org/miekg/dns for all other bits that are needed.

Create a Pull Request from Git in Forgejo

Wed, 28 Jan 2026 06:50:48 +0100

Recently I’ve started to move all my project to Codeberg. This is working great, but one of the things I really missed where the git-push options from GitLab where you can create a PR just by pushing a branch upstream.

Thanks to my whining about this on Mastodon I got this reply about an create-pr-action.

Well, putting this all together, I right now have this script (with little, or no error checking) that does the heavy lifting of creating a PR using Forgejo’s API. This needs an API token with read/write rights on your repo (and only that):

Kong Go CLI manual pages

Fri, 08 Nov 2024 11:38:55 +0100

I’m using Kong the CLI argument parser to create manual pages from the arguments and help described in the Kong structure that tells your CLI how to parse its command line.

With a lot of use from the reflect package (this was before I discovered kong actually has an API you can use), I’m creating manual pages from the kong CLI definition, while having great fun with parsing enum, default and other tags to make the manual page really readable.

Kong Go CLI

Fri, 01 Nov 2024 11:38:55 +0100

I’m a fan of Kong the CLI argument parser, I use it to automatically create manual pages, (bash) completion and do a lot of validation inside it. (This guy also has other really interesting packages, and he’s a huge fan of Go’s struct tags).

I wanted to do validation inside Kong as well, for this Kong has validation, which is just a method (Validate() error) on a type. Now most types I have are just string or any other builtin type. For validation to work, all these types must be re-typed to another type on which the method can be applied, which is annoying. I wanted a better way…

eBPF in Go

Thu, 22 Aug 2024 18:46:57 +0200

What if I could program eBPF in pure Go? I want to use eBPF and lots of current tooling exist, but I like Go. I don’t want to use clang to create the ELF binary that then gets loaded into the Linux kernel. I want to use Go to create that ELF file. The heavy lifting of loading and inspecting can be done via bpftool (although I’m aware that Go programs exist that do this), but bpftool looks like the standard way of interacting with the kernel’s eBPF subsystem.

Script to Upgrade to Latest Go version

Fri, 17 Jul 2020 05:39:00 +0000

This is my super simple script to upgrade to a newer Go version. You might find it handy as well.

#!/bin/zsh

GODIR=~/up

function latestGo {
    local LATEST=$(curl -s 'https://golang.org/dl/?mode=json' | jq -r '.[0].version')
    local INSTALLED=$(go version | awk '{ print $3 }')
    if [[ ${INSTALLED} == ${LATEST} ]]; then
        echo Go is up to date, running ${LATEST} >&2
        exit 0
    fi
    echo Upgrading Go from ${INSTALLED} to ${LATEST} >&2
    local GOLANG=https://dl.google.com/go/${LATEST}.linux-amd64.tar.gz
    local TAR=$(basename $GOLANG)

    ( cd ${GODIR}
      echo Downloading and extracting: $GOLANG >&2
      wget -q $GOLANG && rm -rf go && tar xvfz ${TAR}
    )
}

latestGo

Learning Go

Tue, 13 Aug 2019 07:39:00 +0000

Learning Go is a book that teaches you Go. It includes exercises (and solutions) to help put your new knowledge to the test.

Learning Go in Markdown

Sat, 09 May 2015 07:39:00 +0000

The original Learning Go is written in LaTeX. With LaTeX you get beautiful (PDF) output, but there are a few problems. Most notably I consider LaTeX an output format: it is hard to convert it into something else. Markdown offers much more flexibility in that regard. The downside of markdown is that is not powerful enough to typeset a book. To fix this I forked BlackFriday extended it and renamed it Mmark (These are all written in Golang).

Personal Go install

Mon, 02 Dec 2013 20:28:00 +0000

With golang 1.2, both godoc and go vet are removed from the main repository. Here is a super short list of commands (that I personally use) to install Go and both tools.

export GOBIN=~/bin
export GOPATH=~/g
mkdir -p ~/upstream
mkdir -p ~/g
cd ~/upstream
hg clone https://code.google.com/p/go/
hg update release
cd src
./all.bash
go get code.google.com/p/go.tools/cmd/godoc
go get code.google.com/p/go.tools/cmd/vet
go get code.google.com/p/go.tools/cmd/cover

PKCS11 wrapper for Go

Sat, 17 Aug 2013 03:32:00 +0000

In my quest to write a DNS server in Go I found myself lacking an interface to PKCS#11, so I wrote one. PKCS#11 is the interface to HSMs (including SoftHSM), and even though the interface sucks, I think a DNS server should store its keys in an HSM, by default.

So… here it is. I’m still putting in the finishing touches and some general polish, but it is already usable.

Rfc2pandoc

Wed, 17 Jul 2013 15:04:00 +0000

Have an I-D laying around and sick of typing XML? You want to use Pandoc2rfc, but you don’t want to manually convert the XML (or the generated text) into Pandoc? Well, now finally you can do this automatically. I created a little stylesheet, that can be used to transform the XML to Pandoc. Note the transformation is not perfect, there are a few cases where you still have to edit the Pandoc, most notably:

Pandoc2rfc update(s)

Wed, 03 Jul 2013 17:45:00 +0000

In this post you can find pointers to documentation and source code for the pandoc2rfc package.

Pandoc2rfc source code:: github.com/miekg/pandoc2rfc, if you are not using the shell script (pandoc2rfc), you should just copy transform.xsl and perform the conversions manually (see documentation pointers below)
Packages:: for both xml2rfc and pandoc2rfc can be found at launchpad, these can probably also be installed on Debian and definitely on older Ubuntu versions.
Documentation:: This draft is the canonical documentation.

New Dutch Teletext Script

Tue, 05 Feb 2013 14:17:00 +0000

Last week, the Dutch public broadcaster (NOS) decided to “fix” their online teletext offering, there breaking [an old script I had laying around](/2008/july/15/teletext_nl_script/index.htm l). Their new web interface (displaying gifs) can be found here.

After @bdekruijff discovered their newly, hidden text feed, I decided to adapt the old script to this new situation. Thanks to this page I was able to (partially) reconstruct the feed. Most escape sequences are working, I’m still fiddling with finding Unicode chars for the “Contiguous Graphics Set”.

Pandoc2rfc and xml2rfc

Tue, 22 Jan 2013 12:23:00 +0000

Thanks to the help of Marc Petit-Huguenin I have been able to create (and upload) debian packages (install fine on Ubuntu) for Pandoc2rfc and xml2rfc. For Pandoc2rfc I choose to add a little shell script that calls pandoc and xml2rfc for you. So creating an I-D is as simple as:

pandoc2rfc *.mkd    # or a few other extensions

The shell script depends on transform.xsl to be installed in /usr/lib/pandoc2rfc/, allthough this can be overridden with a flag. When pandoc2rfc is installed you don’t have carry the supports scripts inside your I-D’s source repo. My pandoc2rfc I-D source directory now only has: template.xml, abstract.pdc, back.pdc, middle.pdc and a bib/ directory with references.

Go DNS API change WITH rewrite rules!

Sun, 09 Dec 2012 20:06:00 +0000

In the standard library the DNS types have been renamed from RR_MX to MX which I think is a good change. So I made the same change in Go DNS, but this time I’m providing gofmt -r rewrite rules. They all have the form:

gofmt -r 'RR_A -> A' -w *.go

And then for all the types, so it’s quite a list.

Download the rewrite rules here and use it like $SHELL rewrite.

Adding new RR types to GO DNS

Fri, 30 Nov 2012 20:01:00 +0000

Inspired by NLnet Labs and PowerDNS, I figured I couldn’t stay behind, so here is how to add new RRs to Go DNS.

A small note before I delve into the details, I haven’t optimized Go DNS for adding new types, as this is a relative infrequent event. There are a few items that need to be added before Go DNS understands the new RR type.

Adding the type itself (as a structure) and the four methods needed to implement the RR interface;
Adding the type number and the text string belonging it;
Parsing from text, i.e. when parsing zonefiles.

Lets take the new DANE (RFC6698) as an example. The record is called TLSA, and looks like:

Source Code Pro

Thu, 27 Sep 2012 19:44:00 +0000

Indeed a really nice font and fully open source. Using it now at 11pt. See:

User management in fksd

Tue, 07 Aug 2012 21:58:00 +0000

If you do DNS for too long everything looks like 53.

In this “trace” I’m showing the logging of fksd when I add a zone, try to list it as a non-existent user miekg (which fails), add the user miekg and list it again. User are identified by the key in the TSIG record, their password is the shared secret.

The “config files” from nsupdate can be found in the github repo of fksd. The nsupdate commands are preceded with a %, extra comments are preceded with #:

Dynamic nameserver provisioning with dns pkts

Mon, 06 Aug 2012 23:18:00 +0000

I’m writing a nameserver called fksd (Funkensturm daemon), which is currently in a prototype stage (but the code is available at github).

In this server I’m pursuing some interesting directions in nameserver development, such as the dynamic configuration as provided by BIND10.

BIND10 uses http(s), but I think using DNS packets is more in line with a nameserver, so I opted for that route.

With fksd you can use packets (which will be TSIG signed in the ~~future~~ tomorrow) to configure the server. The only configuration possible at the moment is adding a zone. Such a packet needs to have a TXT record like the following in its AUTHORITY SECTION:

Libunbound wrapper in Go

Sun, 01 Jul 2012 21:37:00 +0000

I’ve created a small wrapper for libunbound for use in Go.

The code can be found at github. It depends on my Go DNS library which can be found here.

Official announcement on the Unbound-users@ list.

To give you a little taste of how it looks, I’ve (re)created tutorials 2 to 6 in Go. Tutorial 2 looks like this, for instance:

package main

// https://www.unbound.net/documentation/libunbound-tutorial-2.html

import (
        "dns"
        "fmt"
        "os"
        "unbound"
)

func main() {
        u := unbound.New()
        defer u.Destroy()

        if err := u.ResolvConf("/etc/resolv.conf"); err != nil {
                fmt.Printf("error %s\n", err.Error())
                os.Exit(1)
        }

        if err := u.Hosts("/etc/hosts"); err != nil {
                fmt.Printf("error %s\n", err.Error())
                os.Exit(1)
        }

        r, err := u.Resolve("www.nlnetlabs.nl.", dns.TypeA, dns.ClassINET)
        if err != nil {
                fmt.Printf("error %s\n", err.Error())
                os.Exit(1)
        }
        fmt.Printf("%+v\n", r)
}

Sync subversion to github.com

Wed, 27 Jun 2012 08:26:00 +0000

This is a post that details on how to sync a subversion repository to git repository on github.com, and how to keep it in sync.

The following sites were instrumental in getting this to work:

There are a number of steps to take. From a bird’s eye view:

Use git svn to clone the svn repo to a git repo;
Create a github git repo;
Add a remote origin in your local git to the remote github repo;
Use some SSH foo to use a separate SSH key for pushing to github.com.

Prerequisites

We are working with the fictional svn repo located at https://svn.example.net/example;
git-svn is installed;
ssh is installed.

Clone the repository

Printing MX records with Go DNS

Thu, 21 Jun 2012 10:30:00 +0000

Now that the API seems to stabilize it is time to update these items.

We want to create a little program that prints out the MX records of domains, like so:

% mx miek.nl
miek.nl.        86400   IN      MX      10 elektron.atoom.net.

% mx microsoft.com 
microsoft.com.  3600    IN      MX      10 mail.messaging.microsoft.com.

We are using my Go DNS package. First the normal header of a Go program, with a bunch of imports. We need the dns package:

draft-gieben-creating-rfcs-pandoc-00.txt

Mon, 04 Jun 2012 09:20:00 +0000

Maybe I’ll try to send it in as a individual submission.



      Network Working Group                                          R. Gieben
      Internet-Draft                                                      SIDN
      Intended status: Informational                                April 2012
      Expires: October 3, 2012


                   Creating Internet Drafts and RFCs using Pandoc
                        draft-gieben-creating-rfcs-pandoc-00

      Abstract

         This memo presents a technique for using Pandoc syntax as a source
         format for documents in the Internet-Drafts (I-Ds) and Request for
         Comments (RFC) series.

         Using Pandoc syntax this way minimizes the need to directly edit the
         raw XML, but it does not completely make the XML invisible.

      Status of this Memo

         This document is an Internet-Draft and is NOT offered in accordance
         with Section 10 of RFC 2026, and the author does not provide the IETF
         with any rights other than to publish as an Internet-Draft.

         Internet-Drafts are working documents of the Internet Engineering
         Task Force (IETF).  Note that other groups may also distribute
         working documents as Internet-Drafts.  The list of current Internet-
         Drafts is at http://datatracker.ietf.org/drafts/current/.

         Internet-Drafts are draft documents valid for a maximum of six months
         and may be updated, replaced, or obsoleted by other documents at any
         time.  It is inappropriate to use Internet-Drafts as reference
         material or to cite them other than as "work in progress."

         This Internet-Draft will expire on October 3, 2012.
















      Gieben                   Expires October 3, 2012                [Page 1]

      Internet-Draft           Pandoc for RFC creation              April 2012


      Table of Contents

         1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . . . 3
         2.  Using Pandoc for RFC creation . . . . . . . . . . . . . . . . . 4
         3.  Syntax  . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
           3.1.  References  . . . . . . . . . . . . . . . . . . . . . . . . 7
         4.  Security Considerations . . . . . . . . . . . . . . . . . . . . 8
         5.  IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 8
         6.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . 8
         7.  Normative References  . . . . . . . . . . . . . . . . . . . . . 8
         Author's Address  . . . . . . . . . . . . . . . . . . . . . . . . . 9








































      Gieben                   Expires October 3, 2012                [Page 2]

      Internet-Draft           Pandoc for RFC creation              April 2012


      1.  Introduction

         This memo presents a technique for using Pandoc [1] syntax as a
         source format for documents in the Internet-Drafts (I-Ds) and Request
         for Comments (RFC) series.

         Pandoc is an "almost plain text" format, which is inspired by
         Markdown Syntax [2] and therefor particularly well suited for editing
         RFC-like documents.

         The power of Pandoc also comes from the fact that it can be
         translated to numerous output formats, including, but not limited to:
         HTML, Markdown and "docbook" XML.

         In this case the Pandoc sources are converted to "docbook" XML.  This
         XML is then converted again, using an XSLT stylesheet, to XML
         suitable as input for "xml2rfc" [RFC2629].  The conversions are
         collectively called Pandoc2rfc [pandoc2rfc].

         Pandoc2rfc is in some way amusing, as we start off with (almost)
         plain text, use elaborate XML and end up with plain text again, as
         shown in Figure 1.

                            Attempt to justify Pandoc2rfc.

              +-------------------+   Pandoc   +---------+
              | ALMOST PLAIN TEXT |   ------>  | DOCBOOK |
              +-------------------+            +---------+
                            |                       |
              non-existent  |                       | XSLT (transform.xsl)
               quicker way  |                       |
                            v                       v
                    +------------+    xml2rfc  +---------+
                    | PLAIN TEXT |  <--------  | XML2RFC |
                    +------------+             +---------+

                                       Figure 1

         For the conversion to work the following tools and files need to be
         installed:

         o  xml2rfc [3];

         o  xsltproc [4] (or any other XSLT (v1) processor);

         o  Pandoc [1];





      Gieben                   Expires October 3, 2012                [Page 3]

      Internet-Draft           Pandoc for RFC creation              April 2012


         o  transform.xsl [5].


      2.  Using Pandoc for RFC creation

         As said in the introduction the use of Pandoc does not eliminate the
         need to setup some files in XML.  Particularly the "<front>" matter
         of "xml2rfc" can not be codified in Pandoc, so a template like this
         is still needed:

                                A minimal template.xml.

           < ?xml version='1.0' ?>
           <!DOCTYPE rfc SYSTEM 'rfc2629.dtd'>

           <rfc ipr='trust200902' docName='draft-gieben-pandoc-rfcs-01'>
            <front>
               <title>Creating Internet Drafts and RFCs using Pandoc</title>
               <abstract>
                   < ?rfc include="abstract.xml"?>
               </abstract>

               <author initials="R." surname="Gieben"
                   fullname="R. (Miek) Gieben">
                   <organization>SIDN</organization>
               </author>

           </front>

           <middle>
               < ?rfc include="middle.xml"?>
           </middle>

           <back>
               <references title="Normative References">
                   < ?rfc include="reference.RFC.2629.xml"?>
               </references>
               < ?rfc include="back.xml"?>
           </back>
           </rfc>

                                       Figure 2

         The template shown in Figure 2 includes 3 (not counting the
         reference) XML files:






      Gieben                   Expires October 3, 2012                [Page 4]

      Internet-Draft           Pandoc for RFC creation              April 2012


         1.  abstract.xml;

         2.  middle.xml;

         3.  back.xml.

         To create the complete document you will need to edit three Pandoc
         files and the template, (".pdc" is the extension for Pandoc files):

         1.  abtract.pdc;

         2.  middle.pdc;

         3.  back.pdc;

         4.  template.xml (probably a fairly static file once setup).

         To convert, for instance, the "middle.pdc" file to XML the following
         command is executed on a Unix-like system:

         pandoc -t docbook -s middle.pdc|xsltproc transform.xsl - > middle.xml

         This is also done for "abstract.pdc" and "back.pdc".  After which
         "xml2rfc" is called:

         xml2rfc template.xml draft.txt

         Which creates the final output.  Of course this process can be
         automated using a tool like "make".

         When using Pandoc2rfc consider adding the following sentence to an
         Acknowledgements section:

         This document was prepared using Pandoc2rfc.


      3.  Syntax

         Almost all features of "xml2rfc" are supported.  A notable exception
         is the "crefs" tag, but HTML comments are allowed within Pandoc
         sources so they may be used as a substitute.

         Sections are started by using a header [README#headers].

         Paragraphs are separated by an empty line.  Hanging paragraphs are
         entered by using a definition list [README#deflists].

         Footnotes are not supported.  Pandoc2rfc (ab)uses the footnote syntax



      Gieben                   Expires October 3, 2012                [Page 5]

      Internet-Draft           Pandoc for RFC creation              April 2012


         to support indices.  Block quotes are not directly supported in
         "xml2rfc" so they get translated to a hanging paragraph.

         A good number of different type of lists are supported, they are
         translated according to the following table.

                      List conversions from Pandoc to "xml2rfc".

          +---------------------------------+------------------------------+
          | Pandoc                          | Converts to                  |
          +---------------------------------+------------------------------+
          | "* First item"                  | "<list style="symbol">"      |
          | "1. First item"                 | "<list style="numbers">"     |
          | "#. First item"                 | "<list style="empty">"       |
          | "a. First item"                 | "<list style="letters">"     |
          | "A. First item"                 | "list style="format %C.">"   |
          | "i. First item"                 | "<list style="format %i.">"  |
          | "I. First item"                 | "<list style="format (%d)">" |
          +---------------------------------+------------------------------+

                                        Table 1

         A figure or artwork is created with a paragraph that is indented with
         four spaces [README#codeblocks].  A figure caption is always
         translated to a "<preamble>".  A figure caption is created by using
         this text as the last line in the artwork: "Figure: ...caption
         text..."

         The different tables [README#tables] Pandoc supports are all mapped
         to "<texttable>".  A table caption is always translated to a
         "<postamble>".  A table caption is added by using "Table: ...caption
         text..." after a table.

         The caption is _always_ translated to a "<preamble>".  The
         "<postamble>" tag isn't supported.  If a table has a caption, it will
         *also* get a reference.  See Section 3.1 for the details.

         As footnotes are not supported in RFCs the syntax in Pandoc is used
         to support an index.  Footnotes in Pandoc (and thus an index in the
         RFC) are entered in two steps, you have a marker in the text, and
         later you give actual footnote text.  Like this:

         [^1]

         [^1]: footnote text

         This text translates to: "<iref item="footnote text"/>".  It points
         to the page where to footnote marker was placed.  Sub items are also



      Gieben                   Expires October 3, 2012                [Page 6]

      Internet-Draft           Pandoc for RFC creation              April 2012


         supported.  Use an exclamation mark ("!") to separate them: "[^1]:
         item!sub item".

      3.1.  References

         References to section are created automatically by Pandoc and the
         normal Pandoc rules are followed.

                    Reference conversions from Pandoc to "xml2rfc".

         +-------------------------+------------------------------+----------+
         | Pandoc                  | Converts to                  | Type     |
         +-------------------------+------------------------------+----------+
         | "[Click](URL)"          | "<eref                       | External |
         |                         | target="URL">Click..."       |          |
         | "[See](#local)"         | "<xref                       | Internal |
         |                         | target="local">See..."       |          |
         | "[](#RFC2119)"          | "<xref target="RFC2119"/>"   | Citation |
         +-------------------------+------------------------------+----------+

                                        Table 2

         Internal references will add "Section:", "Table:" or "Figure:"
         depending on where it points to (this is the default behavior of
         "xml2rfc").  For the citations to work the reference anchor must be
         known (i.e. the RFC reference.xml must be included in the template).

         References to tables and figures are not handled by Pandoc, this
         behavior is implemented in the XSLT stylesheets, therefor the rules
         are slightly different (and less flexible).  A figure and table only
         get a reference when they have a caption.  If a figure has a caption
         it is also centered on the page.

         The reference anchor attribute will be: "fig:" + "first 10
         (normalized) characters from the caption" for figures and "tab:" +
         "first 10 (normalized) characters from the caption" for tables.
         Normalized is:

         o  Take the first 10 characters of the caption (i.e. this is the text
            _after_ the string "Figure:" or "Table:");

         o  Spaces are translated to a minus "-";

         o  Uppercase letters translated to lowercase.

         For example a figure with a caption "Figure: A minimal template" will
         get the anchor "fig:a-minimal-"




      Gieben                   Expires October 3, 2012                [Page 7]

      Internet-Draft           Pandoc for RFC creation              April 2012


      4.  Security Considerations

         This memo raises no security issues.


      5.  IANA Considerations

         This memo has no actions for IANA.


      6.  Acknowledgements

         The following people have helped to make Pandoc2rfc what it is today:
         Benno Overeinder, Erlend Hamnaberg, Matthijs Mekking, and Trygve
         Laugstoel.

         This document was prepared using Pandoc2rfc.


      7.  Normative References

         [README#codeblocks]
                    MacFarlane, J., "PANDOC Documentation", 2006, <http://
                    johnmacfarlane.net/pandoc/
                    README.html#indented-code-blocks>.

         [README#deflists]
                    MacFarlane, J., "PANDOC Documentation", 2006, <http://
                    johnmacfarlane.net/pandoc/README.html#definition-lists>.

         [README#headers]
                    MacFarlane, J., "PANDOC Documentation", 2006,
                    <http://johnmacfarlane.net/pandoc/README.html#headers>.

         [README#tables]
                    MacFarlane, J., "PANDOC Documentation", 2006,
                    <http://johnmacfarlane.net/pandoc/README.html#tables>.

         [RFC2629]  Rose, M., "Writing I-Ds and RFCs using XML", RFC 2629,
                    June 1999.

         [pandoc2rfc]
                    Gieben, R., "Pandoc2rfc", 2012,
                    <http://github.com/miekg/pandoc2rfc>.

         [1]  <http://johnmacfarlane.net/pandoc/>

         [2]  <http://daringfireball.net/projects/markdown/>



      Gieben                   Expires October 3, 2012                [Page 8]

      Internet-Draft           Pandoc for RFC creation              April 2012


         [3]  <http://xml.resource.org/>

         [4]  <http://xmlsoft.org/xslt/xsltproc2.html>

         [5]  <https://raw.github.com/miekg/pandoc2rfc/master/transform.xsl>


      Author's Address

         R. (Miek) Gieben
         SIDN
         Meander 501
         Arnhem,   6825 MD
         NL

         Phone:
         Email: miek@miek.nl
         URI:

































      Gieben                   Expires October 3, 2012                [Page 9]

Super-short guide to getting q (Part II)

Sun, 04 Mar 2012 10:54:00 +0000

The development of the language Go is going at a fast pace, hence an updated version of Super-short guide to gettinq q.

Get the latest version (called weekly) of Go:

Get Go: hg clone -u release https://go.googlecode.com/hg/ go Note the directory you have downloaded it to and set add its bin directory to your PATH: PATH=$PWD/go/bin.
Update Go to the latest weekly: cd go; hg pull; hg update weekly
Compile Go: cd src, you should now sit in go/src. And compile: ./all.bash

godns is now dns

Sat, 11 Feb 2012 23:09:00 +0000

Due to the new go tool (long story), I’ve renamed godns to dns. This means the github repo is also somewhere else.

godns installed itself as dns so code using it does not need to be changed.

DNS Fingerprinting

Sat, 28 Jan 2012 16:03:00 +0000

Announcing FP

The tool for DNS fingerprinting is fpdns, which is Perl based. In recent times development seems to have picked up, but a little competition never hurt anyone, so I wrote fp in Go. Fp is also a fingerprint program for DNS servers. Its aim is to be more readable then fpdns is (was?). And make it more easy to add new server types.

Help needed!

Do you have some old(er) nameserver laying around that can be queried? Does your (sick) mind know of a few extra evil queries that can be sent to nameservers? If so, please contact me: query@evilquery.nl. I want to get to a point where fp sends about 10 queries that can be used to identify a server.

Super-short guide to getting q

Mon, 23 Jan 2012 18:07:00 +0000

Get the latest version (called weekly) of Go:

Get Go: hg clone -u release https://go.googlecode.com/hg/ go Note the directory you have downloaded it to and set $GOROOT to it: export GOROOT=$PWD/go. Add the GOROOT bin directory to your path: PATH=$PATH:$GOROOT/bin
Update Go to the latest weekly: cd $GOROOT; hg pull; hg update weekly
Compile Go: cd $GOROOT/src ; ./all.bash

Install missing commands (gcc, sed, bison, etc.) if needed.

The latest Go is now installed.

XSLT and sectN/section

Sun, 08 Jan 2012 12:31:00 +0000

This came up during a recent Pandoc discussion. The discussion was about outputting <sectN> section styling when creating DocBook XML. Currently Pandoc outputs nested <section>s.

I argued you could easily change between the two formats and <section> is more flexible, so just leave Pandoc as it is. But it allowed me to play with XSLT once more. With the following results.

Translate to sectN

This XSLT translates <section> to <sectN> where N is 5. If the sections are nested deeper it switches to <section>.

Pandoc to RFC

Wed, 28 Sep 2011 13:45:00 +0000

This is an follow-up on this pandoc item in Dutch.

When writing RFC 4641 we directly wrote the XML. Needless to say is was kinda tedious even thought the XML of xml2rfc is very “light”.

Nowadays I’m a fan of the markdown syntax and especially the syntax as supported (created?) by Pandoc.

So for my next RFC (if ever!) I decided I wanted to use Pandoc. As xml2rfc uses XML I thought the easiest way would be to create docbook XML and transform that using XSLT.

Learning Go for E-readers

Sun, 28 Aug 2011 20:45:00 +0000

Thanks to a patch from Thomas Kappler I can now offer two types of PDFs, one for A4 pages and one for E-readers, like the kindle.

The E-reader variant is suffixed with -kindle:

VIM setup

Fri, 12 Aug 2011 09:04:00 +0000

After several years I decided to use a different color scheme for VIM. Also I’m going for force myself to use VIM’s folding abilities and use make from within VIM.

For good measure I also want to use Omni-completion when writing Go code:

Btw, this screenshots also shows the solarized (dark) colorscheme.

Coloring

Google for solarized. In my .vimrc:

let g:solarized_termcolors=256
colorscheme solarized

Make from VIM

Use :make inside the editor and jump through the errors with:

Chaining proxies

Mon, 01 Aug 2011 20:38:00 +0000

Online signing is cool, but slow. Caching queries in a reverse proxy is nice, but useless for something like NSD. But what if you want to do online signing in a fast way?

Enter: proxy chaining.

I already showed FunkenSign (example code is quite old though) and yesterday FunkenShield.

What if you combine the two? That gives the best of both worlds:

Online signing;
Caching;
And it adheres to the true Unix philosophy: do one thing, and do one thing well.

So lets get some figures again.

Reverse DNS proxy

Sun, 31 Jul 2011 16:09:00 +0000

Have a slow nameserver and want to spice things up? How about a reverse DNS proxy? For lack of a cool name I chose the name FunkenShield. It’s (of course) in the early stages, but it works quite nicely already.

This is done with the framework of FunkenSturm. Which is part of GoDNS.

How it works:

You place FunkenShield in front of your nameserver and it will cache the binary packets coming from your server in a local cache.

On programming languages and programmers

Mon, 18 Jul 2011 13:02:00 +0000

Very well written email message from Geoff Teale on the golang mailing list on programmers and progamming languages (thread).

To summarise a long presentation I gave to non-programmers:

There are 12 million programmers in the world

The majority of those programmers are scarcely qualified

Most technology decisions are made by a combination of following the crowd and a false understanding of risk.

The high cost and failure rate in software development is no coincidence.

Remember the Stevie Wonder rule - “When you believe in something you don’t understand then you suffer”. In this case that means “Perhaps making programming language decisions based on what 12 million powerless idiots are doing isn’t the golden road to glory and great hacks.”

Go DNS (update)

Wed, 06 Jul 2011 08:41:00 +0000

I’m finally back to coding Go DNS and making it work with the latest Go releases. Also the API has changed quite significantly since the last time I blogged about it.

So this I will detail key2ds which is small utility that queries a zone and print any DNSKEY records as DS records on the fly, to show the new API and some sample usage.

% ./key2ds sidn.nl
sidn.nl.    0   IN  DS  42033 8 1 343F74674D36C9B5BE2CEB2C401AC4EDEB2A05B2
sidn.nl.    0   IN  DS  42033 8 2 BF985EC0738FACC89EE0B12FBD9261827C59191D9EA6A9BDFF55F9BDF3DBBFF3
sidn.nl.    0   IN  DS  39274 8 1 E79E031DFDE8E68EF1E2C6CA0943C2CC0DED1889
sidn.nl.    0   IN  DS  39274 8 2 8E8A8CFB40FD0C30BFA82E53752E1C257DAFB7B6206D12B9EDA43AF3EAB2157D

This util uses synchronous queries. I will explain the main-function:

DNS reflector server in GO

Mon, 04 Apr 2011 21:08:00 +0000

I’m (again) rewriting the API of godns. I’ve now taking a cue from the http package in Go, the result seems very nice. A reflector (returns your query in the answer) in Go becomes:

package main

import (
        "dns"
        "log"
)

func ReflectServer(w dns.ResponseWriter, req *dns.Msg) {
        m := new(dns.Msg)
        m.SetReply(req)

        m.Extra = make([]dns.RR, 1)
        m.Extra[0] = &dns.RR_TXT{Hdr: dns.RR_Header{Name: m.Question[0].Name, Rrtype: dns.TypeTXT, 
                Class: dns.ClassINET, Ttl: 0}, Txt: "Hello world"}
        buf, _ := m.Pack()
        w.Write(buf)
}

func main() {
        dns.HandleFunc(".", ReflectServer)
        err := dns.ListenAndServe(":8053", "udp", nil)
        if err != nil {
                log.Fatal("ListenAndServe: ", err.String())
        }
}

Learning Go - version 0.3

Mon, 28 Feb 2011 20:54:00 +0000

I’ve tagged a new version of “Learning Go”, version 0.3. It’s more of a snapshot of the current development, than an actual “release” release.

There are lots of fixes, code updates and tweaks. Numerous people send in fixes. Thank you all for the contributions!

Get it:

PDF download;
Latest daily build.

Feedback is welcome.

To see the development version you can use:

Major stuff coming in 0.4:

Funkensign

Mon, 24 Jan 2011 12:52:00 +0000

How about on-the-fly signing? In this example we add a signature to any packet dealing with www.example.org. Again it is a matter of defining the matching, action and setup functions.

Matching

We don’t have to match anything coming in, we only need to sign pkts on their way out. So the function becomes:

func match(m *dns.Msg, d int) (*dns.Msg, bool) {
        // Matching criteria
        switch d { 
        case IN: 
                // nothing
        case OUT:
                // nothing
        }   
        // Packet Mangling
        switch d { 
        case IN: 
                // nothing
        case OUT:
                if m.Question[0].Name == "www.example.org." {
                        // On the way out sign the packet
                        m = sign(m) // keys are global
                }   
        }   
        return m, true
}

As you can see, it calls the sign() function where the actual signing takes place. We just sign the first RR in the answer section — if there is one.

Funkensturm

Sun, 23 Jan 2011 20:59:00 +0000

You need to install Go and Go DNS, but then you can play with Funkensturm:

Funkensturm: delaying proxy example

Sun, 23 Jan 2011 20:50:00 +0000

Another application for Funkensturm is: delaying packets. Here we only delay packets with the recursion desired bit (RD) set, but it can be easily be changed to check for other properties of a packet, see godoc dns for all elements of DNS packets.

The configuration is similar as described here.

Matching pkts with RD bit set

The matching function becomes:

// the only matching we do is on the RD bit
// for incoming packets.
func match(m *dns.Msg, d int) (*dns.Msg, bool) {
        // Matching criteria
        var ok bool
        switch d {
        case IN:
                // only delay pkts with RD bit 
                ok = m.MsgHdr.RecursionDesired == true
        case OUT:
                // nothing
        }

        // Packet Mangling
        switch d {
        case IN:
                // nothing
        case OUT:
                // nothing
        }
        return m, ok
}

Action function

First a delay helper function. As shown here it returns true if the delay time isn’t reached, and false if a something should be delayed.

Funkensturm: transparent proxy example

Sun, 23 Jan 2011 19:00:00 +0000

A transparent proxy is a proxy that does nothing, but it serves as a nice introduction into Funkensturm.

See this post for an architectural overview of Funkensturm.

Currently a configuration is compiled into the Funkensturm binary. As such it must look like a normal Go program.

Matching function

The matching function checks a packet for features and can optionally modify it.

This is the prototype of a matching function:

func match(m *dns.Msg, d int) (*dns.Msg, bool)

It receives a *dns.Msg which is a packet in Go DNS and a direction d which can be IN (first incoming packet) or OUT when the packet is on its way back to the client. It returns a DNS packet and a boolean value signaling a match.

Funkensturm: a versatile DNS proxy

Sun, 23 Jan 2011 16:52:00 +0000

What’s a DNS proxy?

A proxy is something this sits between a client and a server and does something with the data flowing through it. So a DNS proxy does this for DNS packets.

How does Funkensturm work?

Funkensturm consists out of 3 configuration parts:

A setup function for the initial setup.
Match/Modify function(s) — This function checks the packet for configured properties. If there is match true is returned.
Action function(s) — Taking the truth value from 2. (and the packet) as input, it does something with it.

With this setup you can change incoming packets before they are send through to the server. And you can change packet coming from the server that are send back to the client.