In my quest to write a DNS server in Go I found myself lacking an interface to PKCS#11, so I wrote one. PKCS#11 is the interface to HSMs (including SoftHSM), and even though the interface sucks, I think a DNS server should store its keys in an HSM, by default.

So… here it is. I’m still putting in the finishing touches and some general polish, but it is already usable.

Have an I-D laying around and sick of typing XML? You want to use Pandoc2rfc, but you don’t want to manually convert the XML (or the generated text) into Pandoc? Well, now finally you can do this automatically. I created a little stylesheet, that can be used to transform the XML to Pandoc. Note the transformation is not perfect, there are a few cases where you still have to edit the Pandoc, most notably:

In this post you can find pointers to documentation and source code for the pandoc2rfc package.

Pandoc2rfc source code:

github.com/miekg/pandoc2rfc, if you are not using the shell script (pandoc2rfc), you should just copy transform.xsl and perform the conversions manually (see documentation pointers below)

Packages:

for both xml2rfc and pandoc2rfc can be found at launchpad, these can probably also be installed on Debian and definitely on older Ubuntu versions.

Documentation:

This draft is the canonical documentation.

Last week, the Dutch public broadcaster (NOS) decided to “fix” their online teletext offering, there breaking [an old script I had laying around](/2008/july/15/teletext_nl_script/index.htm l). Their new web interface (displaying gifs) can be found here.

After @bdekruijff discovered their newly, hidden text feed, I decided to adapt the old script to this new situation. Thanks to this page I was able to (partially) reconstruct the feed. Most escape sequences are working, I’m still fiddling with finding Unicode chars for the “Contiguous Graphics Set”.

Thanks to the help of Marc Petit-Huguenin I have been able to create (and upload) debian packages (install fine on Ubuntu) for Pandoc2rfc and xml2rfc. For Pandoc2rfc I choose to add a little shell script that calls pandoc and xml2rfc for you. So creating an I-D is as simple as:

pandoc2rfc *.mkd    # or a few other extensions

The shell script depends on transform.xsl to be installed in /usr/lib/pandoc2rfc/, allthough this can be overridden with a flag. When pandoc2rfc is installed you don’t have carry the supports scripts inside your I-D’s source repo. My pandoc2rfc I-D source directory now only has: template.xml, abstract.pdc, back.pdc, middle.pdc and a bib/ directory with references.

In the standard library the DNS types have been renamed from RR_MX to MX which I think is a good change. So I made the same change in Go DNS, but this time I’m providing gofmt -r rewrite rules. They all have the form:

gofmt -r 'RR_A -> A' -w *.go

And then for all the types, so it’s quite a list.

Download the rewrite rules here and use it like $SHELL rewrite.

Inspired by NLnet Labs and PowerDNS, I figured I couldn’t stay behind, so here is how to add new RRs to Go DNS.

A small note before I delve into the details, I haven’t optimized Go DNS for adding new types, as this is a relative infrequent event. There are a few items that need to be added before Go DNS understands the new RR type.

1. Adding the type itself (as a structure) and the four methods needed to implement the RR interface;
2. Adding the type number and the text string belonging it;
3. Parsing from text, i.e. when parsing zonefiles.

Lets take the new DANE (RFC6698) as an example. The record is called TLSA, and looks like:

Indeed a really nice font and fully open source. Using it now at 11pt. See:

• Announcement from Adobe
• Download link

If you do DNS for too long everything looks like 53.

In this “trace” I’m showing the logging of fksd when I add a zone, try to list it as a non-existent user miekg (which fails), add the user miekg and list it again. User are identified by the key in the TSIG record, their password is the shared secret.

The “config files” from nsupdate can be found in the github repo of fksd. The nsupdate commands are preceded with a %, extra comments are preceded with #:

I’m writing a nameserver called fksd (Funkensturm daemon), which is currently in a prototype stage (but the code is available at github).

In this server I’m pursuing some interesting directions in nameserver development, such as the dynamic configuration as provided by BIND10.

BIND10 uses http(s), but I think using DNS packets is more in line with a nameserver, so I opted for that route.

With fksd you can use packets (which will be TSIG signed in the future tomorrow) to configure the server. The only configuration possible at the moment is adding a zone. Such a packet needs to have a TXT record like the following in its AUTHORITY SECTION:

I’ve created a small wrapper for libunbound for use in Go.

The code can be found at github. It depends on my Go DNS library which can be found here.

Official announcement on the Unbound-users@ list.

To give you a little taste of how it looks, I’ve (re)created tutorials 2 to 6 in Go. Tutorial 2 looks like this, for instance:

package main

// https://www.unbound.net/documentation/libunbound-tutorial-2.html

import (
        "dns"
        "fmt"
        "os"
        "unbound"
)

func main() {
        u := unbound.New()
        defer u.Destroy()

        if err := u.ResolvConf("/etc/resolv.conf"); err != nil {
                fmt.Printf("error %s\n", err.Error())
                os.Exit(1)
        }

        if err := u.Hosts("/etc/hosts"); err != nil {
                fmt.Printf("error %s\n", err.Error())
                os.Exit(1)
        }

        r, err := u.Resolve("www.nlnetlabs.nl.", dns.TypeA, dns.ClassINET)
        if err != nil {
                fmt.Printf("error %s\n", err.Error())
                os.Exit(1)
        }
        fmt.Printf("%+v\n", r)
}

This is a post that details on how to sync a subversion repository to git repository on github.com, and how to keep it in sync.

The following sites were instrumental in getting this to work:

• stackoverflow.com question on getting svn into git;
• ssh foo.

There are a number of steps to take. From a bird’s eye view:

1. Use git svn to clone the svn repo to a git repo;
2. Create a github git repo;
3. Add a remote origin in your local git to the remote github repo;
4. Use some SSH foo to use a separate SSH key for pushing to github.com.

Prerequisites

• We are working with the fictional svn repo located at https://svn.example.net/example;
• git-svn is installed;
• ssh is installed.

Clone the repository

Now that the API seems to stabilize it is time to update these items.

We want to create a little program that prints out the MX records of domains, like so:

% mx miek.nl
miek.nl.        86400   IN      MX      10 elektron.atoom.net.

Or

% mx microsoft.com 
microsoft.com.  3600    IN      MX      10 mail.messaging.microsoft.com.

We are using my Go DNS package. First the normal header of a Go program, with a bunch of imports. We need the dns package:

Maybe I’ll try to send it in as a individual submission.

• Source repository at github
• As txt download
• Pandoc2rfc code repository

      Network Working Group                                          R. Gieben
      Internet-Draft                                                      SIDN
      Intended status: Informational                                April 2012
      Expires: October 3, 2012


                   Creating Internet Drafts and RFCs using Pandoc
                        draft-gieben-creating-rfcs-pandoc-00

      Abstract

         This memo presents a technique for using Pandoc syntax as a source
         format for documents in the Internet-Drafts (I-Ds) and Request for
         Comments (RFC) series.

         Using Pandoc syntax this way minimizes the need to directly edit the
         raw XML, but it does not completely make the XML invisible.

      Status of this Memo

         This document is an Internet-Draft and is NOT offered in accordance
         with Section 10 of RFC 2026, and the author does not provide the IETF
         with any rights other than to publish as an Internet-Draft.

         Internet-Drafts are working documents of the Internet Engineering
         Task Force (IETF).  Note that other groups may also distribute
         working documents as Internet-Drafts.  The list of current Internet-
         Drafts is at http://datatracker.ietf.org/drafts/current/.

         Internet-Drafts are draft documents valid for a maximum of six months
         and may be updated, replaced, or obsoleted by other documents at any
         time.  It is inappropriate to use Internet-Drafts as reference
         material or to cite them other than as "work in progress."

         This Internet-Draft will expire on October 3, 2012.
















      Gieben                   Expires October 3, 2012                [Page 1]

      Internet-Draft           Pandoc for RFC creation              April 2012


      Table of Contents

         1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . . . 3
         2.  Using Pandoc for RFC creation . . . . . . . . . . . . . . . . . 4
         3.  Syntax  . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
           3.1.  References  . . . . . . . . . . . . . . . . . . . . . . . . 7
         4.  Security Considerations . . . . . . . . . . . . . . . . . . . . 8
         5.  IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 8
         6.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . 8
         7.  Normative References  . . . . . . . . . . . . . . . . . . . . . 8
         Author's Address  . . . . . . . . . . . . . . . . . . . . . . . . . 9








































      Gieben                   Expires October 3, 2012                [Page 2]

      Internet-Draft           Pandoc for RFC creation              April 2012


      1.  Introduction

         This memo presents a technique for using Pandoc [1] syntax as a
         source format for documents in the Internet-Drafts (I-Ds) and Request
         for Comments (RFC) series.

         Pandoc is an "almost plain text" format, which is inspired by
         Markdown Syntax [2] and therefor particularly well suited for editing
         RFC-like documents.

         The power of Pandoc also comes from the fact that it can be
         translated to numerous output formats, including, but not limited to:
         HTML, Markdown and "docbook" XML.

         In this case the Pandoc sources are converted to "docbook" XML.  This
         XML is then converted again, using an XSLT stylesheet, to XML
         suitable as input for "xml2rfc" [RFC2629].  The conversions are
         collectively called Pandoc2rfc [pandoc2rfc].

         Pandoc2rfc is in some way amusing, as we start off with (almost)
         plain text, use elaborate XML and end up with plain text again, as
         shown in Figure 1.

                            Attempt to justify Pandoc2rfc.

              +-------------------+   Pandoc   +---------+
              | ALMOST PLAIN TEXT |   ------>  | DOCBOOK |
              +-------------------+            +---------+
                            |                       |
              non-existent  |                       | XSLT (transform.xsl)
               quicker way  |                       |
                            v                       v
                    +------------+    xml2rfc  +---------+
                    | PLAIN TEXT |  <--------  | XML2RFC |
                    +------------+             +---------+

                                       Figure 1

         For the conversion to work the following tools and files need to be
         installed:

         o  xml2rfc [3];

         o  xsltproc [4] (or any other XSLT (v1) processor);

         o  Pandoc [1];





      Gieben                   Expires October 3, 2012                [Page 3]

      Internet-Draft           Pandoc for RFC creation              April 2012


         o  transform.xsl [5].


      2.  Using Pandoc for RFC creation

         As said in the introduction the use of Pandoc does not eliminate the
         need to setup some files in XML.  Particularly the "<front>" matter
         of "xml2rfc" can not be codified in Pandoc, so a template like this
         is still needed:

                                A minimal template.xml.

           < ?xml version='1.0' ?>
           <!DOCTYPE rfc SYSTEM 'rfc2629.dtd'>

           <rfc ipr='trust200902' docName='draft-gieben-pandoc-rfcs-01'>
            <front>
               <title>Creating Internet Drafts and RFCs using Pandoc</title>
               <abstract>
                   < ?rfc include="abstract.xml"?>
               </abstract>

               <author initials="R." surname="Gieben"
                   fullname="R. (Miek) Gieben">
                   <organization>SIDN</organization>
               </author>

           </front>

           <middle>
               < ?rfc include="middle.xml"?>
           </middle>

           <back>
               <references title="Normative References">
                   < ?rfc include="reference.RFC.2629.xml"?>
               </references>
               < ?rfc include="back.xml"?>
           </back>
           </rfc>

                                       Figure 2

         The template shown in Figure 2 includes 3 (not counting the
         reference) XML files:






      Gieben                   Expires October 3, 2012                [Page 4]

      Internet-Draft           Pandoc for RFC creation              April 2012


         1.  abstract.xml;

         2.  middle.xml;

         3.  back.xml.

         To create the complete document you will need to edit three Pandoc
         files and the template, (".pdc" is the extension for Pandoc files):

         1.  abtract.pdc;

         2.  middle.pdc;

         3.  back.pdc;

         4.  template.xml (probably a fairly static file once setup).

         To convert, for instance, the "middle.pdc" file to XML the following
         command is executed on a Unix-like system:

         pandoc -t docbook -s middle.pdc|xsltproc transform.xsl - > middle.xml

         This is also done for "abstract.pdc" and "back.pdc".  After which
         "xml2rfc" is called:

         xml2rfc template.xml draft.txt

         Which creates the final output.  Of course this process can be
         automated using a tool like "make".

         When using Pandoc2rfc consider adding the following sentence to an
         Acknowledgements section:

         This document was prepared using Pandoc2rfc.


      3.  Syntax

         Almost all features of "xml2rfc" are supported.  A notable exception
         is the "crefs" tag, but HTML comments are allowed within Pandoc
         sources so they may be used as a substitute.

         Sections are started by using a header [README#headers].

         Paragraphs are separated by an empty line.  Hanging paragraphs are
         entered by using a definition list [README#deflists].

         Footnotes are not supported.  Pandoc2rfc (ab)uses the footnote syntax



      Gieben                   Expires October 3, 2012                [Page 5]

      Internet-Draft           Pandoc for RFC creation              April 2012


         to support indices.  Block quotes are not directly supported in
         "xml2rfc" so they get translated to a hanging paragraph.

         A good number of different type of lists are supported, they are
         translated according to the following table.

                      List conversions from Pandoc to "xml2rfc".

          +---------------------------------+------------------------------+
          | Pandoc                          | Converts to                  |
          +---------------------------------+------------------------------+
          | "* First item"                  | "<list style="symbol">"      |
          | "1. First item"                 | "<list style="numbers">"     |
          | "#. First item"                 | "<list style="empty">"       |
          | "a. First item"                 | "<list style="letters">"     |
          | "A. First item"                 | "list style="format %C.">"   |
          | "i. First item"                 | "<list style="format %i.">"  |
          | "I. First item"                 | "<list style="format (%d)">" |
          +---------------------------------+------------------------------+

                                        Table 1

         A figure or artwork is created with a paragraph that is indented with
         four spaces [README#codeblocks].  A figure caption is always
         translated to a "<preamble>".  A figure caption is created by using
         this text as the last line in the artwork: "Figure: ...caption
         text..."

         The different tables [README#tables] Pandoc supports are all mapped
         to "<texttable>".  A table caption is always translated to a
         "<postamble>".  A table caption is added by using "Table: ...caption
         text..." after a table.

         The caption is _always_ translated to a "<preamble>".  The
         "<postamble>" tag isn't supported.  If a table has a caption, it will
         *also* get a reference.  See Section 3.1 for the details.

         As footnotes are not supported in RFCs the syntax in Pandoc is used
         to support an index.  Footnotes in Pandoc (and thus an index in the
         RFC) are entered in two steps, you have a marker in the text, and
         later you give actual footnote text.  Like this:

         [^1]

         [^1]: footnote text

         This text translates to: "<iref item="footnote text"/>".  It points
         to the page where to footnote marker was placed.  Sub items are also



      Gieben                   Expires October 3, 2012                [Page 6]

      Internet-Draft           Pandoc for RFC creation              April 2012


         supported.  Use an exclamation mark ("!") to separate them: "[^1]:
         item!sub item".

      3.1.  References

         References to section are created automatically by Pandoc and the
         normal Pandoc rules are followed.

                    Reference conversions from Pandoc to "xml2rfc".

         +-------------------------+------------------------------+----------+
         | Pandoc                  | Converts to                  | Type     |
         +-------------------------+------------------------------+----------+
         | "[Click](URL)"          | "<eref                       | External |
         |                         | target="URL">Click..."       |          |
         | "[See](#local)"         | "<xref                       | Internal |
         |                         | target="local">See..."       |          |
         | "[](#RFC2119)"          | "<xref target="RFC2119"/>"   | Citation |
         +-------------------------+------------------------------+----------+

                                        Table 2

         Internal references will add "Section:", "Table:" or "Figure:"
         depending on where it points to (this is the default behavior of
         "xml2rfc").  For the citations to work the reference anchor must be
         known (i.e. the RFC reference.xml must be included in the template).

         References to tables and figures are not handled by Pandoc, this
         behavior is implemented in the XSLT stylesheets, therefor the rules
         are slightly different (and less flexible).  A figure and table only
         get a reference when they have a caption.  If a figure has a caption
         it is also centered on the page.

         The reference anchor attribute will be: "fig:" + "first 10
         (normalized) characters from the caption" for figures and "tab:" +
         "first 10 (normalized) characters from the caption" for tables.
         Normalized is:

         o  Take the first 10 characters of the caption (i.e. this is the text
            _after_ the string "Figure:" or "Table:");

         o  Spaces are translated to a minus "-";

         o  Uppercase letters translated to lowercase.

         For example a figure with a caption "Figure: A minimal template" will
         get the anchor "fig:a-minimal-"




      Gieben                   Expires October 3, 2012                [Page 7]

      Internet-Draft           Pandoc for RFC creation              April 2012


      4.  Security Considerations

         This memo raises no security issues.


      5.  IANA Considerations

         This memo has no actions for IANA.


      6.  Acknowledgements

         The following people have helped to make Pandoc2rfc what it is today:
         Benno Overeinder, Erlend Hamnaberg, Matthijs Mekking, and Trygve
         Laugstoel.

         This document was prepared using Pandoc2rfc.


      7.  Normative References

         [README#codeblocks]
                    MacFarlane, J., "PANDOC Documentation", 2006, <http://
                    johnmacfarlane.net/pandoc/
                    README.html#indented-code-blocks>.

         [README#deflists]
                    MacFarlane, J., "PANDOC Documentation", 2006, <http://
                    johnmacfarlane.net/pandoc/README.html#definition-lists>.

         [README#headers]
                    MacFarlane, J., "PANDOC Documentation", 2006,
                    <http://johnmacfarlane.net/pandoc/README.html#headers>.

         [README#tables]
                    MacFarlane, J., "PANDOC Documentation", 2006,
                    <http://johnmacfarlane.net/pandoc/README.html#tables>.

         [RFC2629]  Rose, M., "Writing I-Ds and RFCs using XML", RFC 2629,
                    June 1999.

         [pandoc2rfc]
                    Gieben, R., "Pandoc2rfc", 2012,
                    <http://github.com/miekg/pandoc2rfc>.

         [1]  <http://johnmacfarlane.net/pandoc/>

         [2]  <http://daringfireball.net/projects/markdown/>



      Gieben                   Expires October 3, 2012                [Page 8]

      Internet-Draft           Pandoc for RFC creation              April 2012


         [3]  <http://xml.resource.org/>

         [4]  <http://xmlsoft.org/xslt/xsltproc2.html>

         [5]  <https://raw.github.com/miekg/pandoc2rfc/master/transform.xsl>


      Author's Address

         R. (Miek) Gieben
         SIDN
         Meander 501
         Arnhem,   6825 MD
         NL

         Phone:
         Email: miek@miek.nl
         URI:

































      Gieben                   Expires October 3, 2012                [Page 9]

The development of the language Go is going at a fast pace, hence an updated version of Super-short guide to gettinq q.

Get the latest version (called weekly) of Go:

1. Get Go: hg clone -u release https://go.googlecode.com/hg/ go Note the directory you have downloaded it to and set add its bin directory to your PATH: PATH=$PWD/go/bin.

2. Update Go to the latest weekly: cd go; hg pull; hg update weekly

3. Compile Go: cd src, you should now sit in go/src. And compile: ./all.bash

Due to the new go tool (long story), I’ve renamed godns to dns. This means the github repo is also somewhere else.

godns installed itself as dns so code using it does not need to be changed.

Announcing FP

The tool for DNS fingerprinting is fpdns, which is Perl based. In recent times development seems to have picked up, but a little competition never hurt anyone, so I wrote fp in Go. Fp is also a fingerprint program for DNS servers. Its aim is to be more readable then fpdns is (was?). And make it more easy to add new server types.

Help needed!

Do you have some old(er) nameserver laying around that can be queried? Does your (sick) mind know of a few extra evil queries that can be sent to nameservers? If so, please contact me: query@evilquery.nl. I want to get to a point where fp sends about 10 queries that can be used to identify a server.

Get the latest version (called weekly) of Go:

1. Get Go: hg clone -u release https://go.googlecode.com/hg/ go Note the directory you have downloaded it to and set $GOROOT to it: export GOROOT=$PWD/go. Add the GOROOT bin directory to your path: PATH=$PATH:$GOROOT/bin

2. Update Go to the latest weekly: cd $GOROOT; hg pull; hg update weekly

3. Compile Go: cd $GOROOT/src ; ./all.bash

   Install missing commands (gcc, sed, bison, etc.) if needed.

The latest Go is now installed.

This came up during a recent Pandoc discussion. The discussion was about outputting <sectN> section styling when creating DocBook XML. Currently Pandoc outputs nested <section>s.

I argued you could easily change between the two formats and <section> is more flexible, so just leave Pandoc as it is. But it allowed me to play with XSLT once more. With the following results.

Translate to sectN

This XSLT translates <section> to <sectN> where N is 5. If the sections are nested deeper it switches to <section>.

This is an follow-up on this pandoc item in Dutch.

When writing RFC 4641 we directly wrote the XML. Needless to say is was kinda tedious even thought the XML of xml2rfc is very “light”.

Nowadays I’m a fan of the markdown syntax and especially the syntax as supported (created?) by Pandoc.

So for my next RFC (if ever!) I decided I wanted to use Pandoc. As xml2rfc uses XML I thought the easiest way would be to create docbook XML and transform that using XSLT.

Thanks to a patch from Thomas Kappler I can now offer two types of PDFs, one for A4 pages and one for E-readers, like the kindle.

The E-reader variant is suffixed with -kindle:

• Learning Go for E-readers
• Learning Go A4 paper

After several years I decided to use a different color scheme for VIM. Also I’m going for force myself to use VIM’s folding abilities and use make from within VIM.

For good measure I also want to use Omni-completion when writing Go code:

Btw, this screenshots also shows the solarized (dark) colorscheme.

Coloring

Google for solarized. In my .vimrc:

let g:solarized_termcolors=256
colorscheme solarized

Make from VIM

Use :make inside the editor and jump through the errors with:

Online signing is cool, but slow. Caching queries in a reverse proxy is nice, but useless for something like NSD. But what if you want to do online signing in a fast way?

Enter: proxy chaining.

I already showed FunkenSign (example code is quite old though) and yesterday FunkenShield.

What if you combine the two? That gives the best of both worlds:

• Online signing;
• Caching;
• And it adheres to the true Unix philosophy: do one thing, and do one thing well.

So lets get some figures again.