Coding – wenzr

With Go, the filepath Glob() function is a really fast way to find all files that match a particular path, with optional wild cards. I was disappointed, though, that there is not a way to force case-insensitivity to be performed by the Glob() function to the entire pattern that is passed in.

What I found was that the Glob() function is case-insensitive on Windows and case-sensitive on Linux, due to the way that each OS handles file paths. With a regex pattern you can easily force case-insensitivity to the entire pattern using (?i), unfortunately this doesn’t work with Glob() because it doesn’t use regex patterns. Don’t give up hope, though, there is a way to do this.

The Glob() function allows the use of the ‘[‘ and ‘]’ runes to define patterns within the pattern string that is passed into the Glob() function. So the best way I found to do a case-insensitive search is to replace every Letter category Unicode code-point (aka rune) with a lowercase and uppercase version of the rune surrounded by square brackets. So ‘a’ would become ‘[aA]’ and ‘test’ would become ‘[tT][eE][sS][tT]’. In case you don’t know what is included in the Unicode Letter category, you can view the contents here. I believe that the Letter category is made up of the combination of the Ll, Lm, Lo, Lt, and Lu categories. There is no need to alter the path on Windows, since it is already case-insensitive, so I wrote a function to only be performed if the OS is not Windows.

Here is the Go code that I used to perform a conversion of a file path string into a case-insensitive version of the same file path:

func InsensitiveFilepath(path string) string {
   if runtime.GOOS == "windows" {
      return path
   }

   p := ""
   for _, r := range path {
      if unicode.IsLetter(r) {
         p += fmt.Sprintf("[%c%c]", unicode.ToLower(r), unicode.ToUpper(r))
      } else {
         p += string(r)
      }
   }
   return p
}

Here is what it looks like when it runs:

fmt.Println(InsensitiveFilepath("/home/user/test/*"))

/[hH][oO][mM][eE]/[uU][sS][eE][rR]/[tT][eE][sS][tT]/*

You can play with it yourself on the the Go playground!

I wrote an application in Go, aka Golang, that I use to search for text matching a regex pattern within text files. Up to now, everything has worked well with my testing on Windows 10. However, I put my compiled application on to a Linux machine (specifically RHEL) today and I was getting errors with my os.MkdirAll() call. The function call looked like this in my code:

if _, err := os.Stat(logDirectory); os.IsNotExist(err) {
   os.MkdirAll(logDirectory, 0777)
}

Basically this code creates a directory using the path specified in the variable logDirectory with the permissions 777 (Read, Write, and Execute for all users) if the directory does not already exist. If the directory does not exist it will create the directory as well as any necessary parent directories that don’t already exist.

What I found is that my application creates the non-existent directory, but the permissions specified (0777) aren’t what are actually applied to the newly created directory. What ends up being applied is the default permissions according to the umask that is set. So you can see below that my umask is set to 0777 (meaning grant no permissions to newly created files and directories) so the newly created directory has the permissions 0000.

If you are unfamiliar with umask, like I was, then here is a basic description as I understand it. Umask is the default permission given when a new file or folder is created on a Linux/Unix machine. Umask accepts values between 0 and 7 that are the negated form of normal permissions. Meaning if you want the default permissions on a directory to be 765 then you should set the umask value to 012. Or if you want the default permissions to be 555, then you should set umask to 222. You can set umask in a shell simply by typing ‘umask’ followed by the desired value. You can view what umask is set to by entering ‘umask’ with no value after it. For example:

So back to the issue my Go application was experiencing. I thought this must be some sort of bug when running Go on Linux. Through some Googling, I found that many other Go users have thought the same thing as me. However, the creators of Go state that this is not a bug. Basically their answer is that Go is built to run on various OS’s, but it still has to honor the behaviors expected on each OS. Meaning that in my case, I should not expect file permissions to be handled consistently between my testing on Windows 10 and Linux RHEL. This makes sense. Although, it makes me wonder why the Go MkdirAll() function accepts a parameter for setting the permissions if it isn’t going to be honored.

The Solution

The solution I was able to come up with to solve this issue is that you must use the os.Chmod() function to set the permissions of a directory after creating the directory with the os.MkdirAll() function. Yes it seems weird that I am stating in code to set the permissions to 0777 twice, but that’s the way it is. So here is my updated code:

if _, err := os.Stat(logDirectory); os.IsNotExist(err) {
   os.MkdirAll(logDirectory, 0777)
   os.Chmod(logDirectory, 0777)
}

Here are the results of my test on Linux using this new code:

As you can see, umask was set to 0777 which means that a newly created directory should have the default permissions of 0000. However, thanks to the addition of os.Chmod() the logs directory now has the permissions 0777. Now that the logs directory has the desired permissions, my Go application is able to run as expected and create new log files when needed.