Open Source file readers

- 2 minutes read - 284 words

Hi All,

I would like to start my posting journey by talking about an issue that is near and dear to my heart: reading data.

In Australia lucky geologists and data enthusiasts have access to a fantastic source of open file geoscientific data provided by our geological services such as GSWA and as tenements are released the data from then slowly becomes available as open file data under the fantastic CC4.0 license.

On occasion we are provided data that unfortunately can be difficult to read and then interpret. For the sake of argument I’ve listed the 3 most common source of issues I’ve come across before:

  1. proprietary encodings (binary)
  2. structured text with headers forming important metadata
  3. data packages, files of various types in folders

Apart from the DMIRS data we also have access to an amazing set of information in AuScope which has a large collection of data ranging from Hylogger data to something that I had heard of but had never analysed before Tescan Tema data.

So with data downloaded in hand I thought that this would be a great opportunity to demonstrate some of the technical services that I could provide such as particle sizing, shape analysis, or mineral associations.

Unfortunately this goal was not as straight forward as intended as the TIMA MINDIF (MINeralogy Data Interchange format) needed a little…decoding. After a little googling and duck duck going I found that there is no python library to decode the MINDIF format. So in the spirit of open data I have released the file reader that I have written under an MIT license and can be found here at pytima hopefully this will be of use to the wider community.

Thanks,

Ben.