In my previous post, I talked about getting a very basic static file server up and running on openshift using Go.

I was unhappy with several things:

  • Automatic directory listings: I do not want them
  • It serves up "dot-files": I don't want that
  • It should automatically redirect to https if requests come in over http

So, with limited experience, I played around to find a way to do this with Go. Not perfect, but good enough for my needs.

Changing http.FileServer's default behaviour

In Go, there is a built in HTTP server for static files: http.FileServer, as discussed in the previous part.

Basically, it implements Go's [http.Handler interface](https://golang.org/pkg/net/http/#Handler http.Handler).

But it has no built in way to filter out certain files or disable directory listings that I know of.

So to deal with this I made my own Handler implementations to do these tasks,
and only pass it through to the http.FileServer when I really want it to serve up some content.

From the previous post, we got this code:

// Get the ip address
ip := os.Getenv("HOST")
// Get the port to bind to
port := os.Getenv("PORT")
// Get the location where the public files are ...
public := os.Getenv("OPENSHIFT_DATA_DIR")
// Get the bind address ...
bindAddress := fmt.Sprintf("%s:%s", ip, port)
// Set up the server
fileHandler := http.FileServer(http.Dir(public))
http.ListenAndServe(bindAddress, fileHandler)

As you can see, we create the http.FileServer and pass it as the handler to the http.ListenAndServe call (which "listens to and serves" HTTP requests / responses).

Instead of passing it the http.FileServer handler, you can pass it your own handler and make that handler call the http.FileServer.

Here's a function that returns such a handler:

func protectFiles(h http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        allowed := checkIfRequestIsAllowed(r)
        if allowed {
            h.ServeHTTP(w, r)
        } else {
            http.NotFound(w, r)
        }
    })
}

The "protectFiles" method is passed an http.Handler. Basically, when it decides it's OK to handle the request, it will pass it through to that handler. However, when it decides it's not OK, it will serve up a 404 response and never call the "next" handler.

In this manner, you can create a chain of http handlers, each making their own decisions.

The "checkIfRequestIsAllowed" does some things, but that's not important right now. The point I was making is that I'm chaining handlers together and only calling http.FileServer when I really want to serve up files.

Not sure if this is the perfect way, or "idiomatic go", but it's what I did to make it work.

So to implement the functions I wanted, I had to create three such handlers:

  • To disable directory listings
  • To disable serving "dot-files"
  • To anwser HTTP requests with a redirect to HTTPS

Disabling directory listings

For this one, I needed to check if the file that was requested is a directory or not. So first I need the full path of the file that is being requested. Since the http request path is relative to the root directory of my fileServer, I had to construct this full path first.

The root, I know, is my OPENSHIFT_DATA_DIR.

The request I get from the http.listenAndServe routine.

// Helper function that will look at the Path of the URL and to root
// of the public files and return the full path of the requested file.
func getFullPathFromRequest(r *http.Request, root string) string {
    fullPath := path.Join(root, r.URL.Path)
    return fullPath
}

Using this full path, I can make a function that decides whether or not to serve the request::

func ValidateDir(fullPath string) bool {
    if file, err := os.Stat(fullPath); err == nil {
        if file.IsDir() {
            // File is a directory, so we won't validate it.
            return false
        } else {
            // Non-directory file, we are OK to validate this.
            return true
        }
    }
    // If the file info cannot be retrieved, do not allow the
    // file to be served, so do not validate it ...
    return false
}

Of course, I need to have something that creates the custom handler also:

// Creates an http handler that will return a 404 if a directory listing
// is requested
func protectFiles(h http.Handler, filters []Filter, root string) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        allowed := true
        fullPath := getFullPathFromRequest(r, root)
        allowed := ValidateDir(fullPath);
        if allowed {
            h.ServeHTTP(w, r)
        } else {
            http.NotFound(w, r)
        }
    })
}

Do not serve dot-files

In a very similar manner, using the full path, I can see if a file is a dot-file, and then I do not validate the request:

// Check if the path refers to a file or directory whose
// names starts with a .
func ValidateDotFiles(fullPath string) bool {
    basePath := path.Base(fullPath)
    isPrivate := (strings.HasPrefix(basePath, "."))
    if isPrivate {
        return false
    } else {
        return true
    }
}

I can call this ValidateDotFiles together with the ValidateDir function in the same protectFiles handler.

One thing missing here: if a non-dot-file is inside a "dot-file" directory, it will still be validated and served. So this function should really traverse the entire directory tree to see if any of the parent directory's names start with a dot.

But I'm not doing that, yet.

Redirect to HTTPS

Since openshift exposes my website via a reverse proxy (which accepts HTTPS), technically all requests are coming into my webserver over HTTP. So I didn't have to spool in a certificate and make Go serve HTTPS (it can ...) because I don't have a valid certificate nor a domain name of my own.

So basically, the request comes in to files-aaronlenoir.rhcloud.com over HTTPS and openshift forwards the request internally via an HTTP call.

So how do I know that someone was accessing my server via HTTP? All requests look the same.

In this case, I have to check the "X-Forwarded-Proto" HTTP header. If it contains "https" as the value, we're good, if it contains "http", the request came into the reverse proxy over http. In the latter case, I need to redirect to my website over https.

The creation of the complete handler looks like this:

// Creates an http handler that will check if the request came in over
// http or https and will force the user to https, additionally it will
// add the HSTS header.
func forceHttps(h http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        forwardedProto := r.Header["X-Forwarded-Proto"]
        if len(forwardedProto) == 0 || forwardedProto[0] != "https" {
            // None HTTPS request, forward to https ...
            target := fmt.Sprintf("https://%s%s", r.Host, r.RequestURI)
            http.Redirect(w, r, target, 301)
        } else {
            // Working in HTTPS, add HSTS header and serve content.
            w.Header().Set("Strict-Transport-Security", "max-age=31536000")
            h.ServeHTTP(w, r)
        }
    })
}

For good measure, I add an HSTS header as well.

Putting it all together

We can expand the example of the first post as follows:

// Get the ip address
ip := os.Getenv("HOST")
// Get the port to bind to
port := os.Getenv("PORT")
// Get the location where the public files are ...
public := os.Getenv("OPENSHIFT_DATA_DIR")
// Get the bind address ...
bindAddress := fmt.Sprintf("%s:%s", ip, port)
// Set up the server
fileHandler := http.FileServer(http.Dir(public))
// protectFileHandler will forward the request to fileHandler
// if allowed.
protectFileHandler := protectFiles(fileHandler)
// forceHttpsHandler will forward the request to protectFileHandler
// if ok.
forceHttpsHandler := forceHttps(protectFileHandler)
// ListenAndServe will pass all requests first to forceHttpsHandler
http.ListenAndServe(bindAddress, forceHttpsHandler)

In the next part, I'll describe how I tried to wrap this all up in a separate module.