This is CallMeAlan.uk

Your Photographs and EXIF - a mass of data

A little upfront info
Providing you have a decent camera it will record an unbelievable amount of data right there in the file, along with the picture itself. I can't define what I mean by 'decent' there, not having researched this particular point, but I suspect that even the most humble little point 'n' shoot will feature the ability to record at least some EXIF data. EXIF stands for Exchangeable image file format, a standard that specifies the formats for images, sound, and ancillary tags used by digital cameras. At its utter simplest level EXIF records aperture, shutter speed and ISO. Right-click on one of your pictures and under Properties you should be able to see those values. But with a software tool called EXIFTOOL you can access hundreds of indicators and values.

Me and my photography
Photography fascinates me; I own various cameras, both still and movie. Some are hand-held, others go aloft, such as those in my two quadcopters. There's a pretty decent little camera in my iPhone 6 Plus, capable of producing remarkably good quick snapshots if nothing else is to hand and, incidentally, rendered even better if you load up software which enables you to shoot RAW and later post-process in order to extract the ultimate quality. one such app is SilkyPix Shot, free. Check it out.

But my main workhorse is my Pentax K-1, which is full-frame and delivers 36 megapixel results. I have three splendid lenses to accompany the camera, an ultra-wide 14mm, a standard 50mm and a macro/modest telephoto 100mm.

My EXIF realisation
So, as a keen photographer I'm a keen follower of DPReview, where I can read all the latest photography news and dip into their forums. It was in a message in one of the forums quite recently where somebody asked for advice upon extracting some piece of information from his camera's EXIFs. This piqued my interest and I did a bit of research on the matter, quickly discovering that there's a software utility called EXIFTOOL which can read (and, incidentally, write) EXIF material from pictures. I was amazed to see how much data is in a picture!

By way of an example, here is the EXIF from a recent photograph of mine.

ExifTool Version Number : 10.56
File Name : ALAN4918.jpg
Directory : /Users/[.......]
File Size : 25 MB
Resource Fork Size : 506 kB
File Modification Date/Time : 2017:06:19 08:20:33+01:00
File Access Date/Time : 2017:06:22 09:53:58+01:00
File Inode Change Date/Time : 2017:06:19 08:20:33+01:00
File Permissions : rw-r--r--
File Type : JPEG
File Type Extension : jpg
MIME Type : image/jpeg
JFIF Version : 1.01
Exif Byte Order : Little-endian (Intel, II)
Make : RICOH IMAGING COMPANY, LTD.
Camera Model Name : PENTAX K-1
X Resolution : 300
Y Resolution : 300
Resolution Unit : inches
Software : Capture One 10 Macintosh
Exposure Time : 1/160
F Number : 4.0
Exposure Program : Not Defined
ISO : 125
Sensitivity Type : Standard Output Sensitivity
Standard Output Sensitivity : 125
Exif Version : 0230
Date/Time Original : 2017:06:18 11:43:18
Create Date : 2017:06:18 11:43:18
Shutter Speed Value : 1/160
Aperture Value : 4.0
Exposure Compensation : -0.3
Metering Mode : Spot
Flash : Off, Did not fire
Focal Length : 50.0 mm
Exif Image Width : 7360
Exif Image Height : 4912
Sensing Method : One-chip color area
File Source : Digital Camera
Scene Type : Directly photographed
Custom Rendered : Custom
Exposure Mode : Manual
White Balance : Manual
Focal Length In 35mm Format : 50 mm
Scene Capture Type : Standard
Contrast : Normal
Saturation : Low
Sharpness : Hard
Subject Distance Range : Macro
Lens Model : 50 mm
GPS Version ID : 2.3.0.0
GPS Altitude Ref : Above Sea Level
GPS Time Stamp : 11:43:18
GPS Satellites : 08
GPS Status : Measurement Active
GPS Measure Mode : 3-Dimensional Measurement
GPS Speed Ref : km/h
GPS Speed : 0.07
GPS Track Ref : True North
GPS Track : 300.17
GPS Img Direction Ref : True North
GPS Img Direction : 237.17
GPS Map Datum : WGS-84
GPS Processing Method : GPS
GPS Date Stamp : 2017:06:18
Compression : JPEG (old-style)
Thumbnail Offset : 1128
Thumbnail Length : 2689
Profile CMM Type : ADBE
Profile Version : 2.1.0
Profile Class : Display Device Profile
Color Space Data : RGB
Profile Connection Space : XYZ
Profile Date Time : 2000:08:11 19:51:59
Profile File Signature : acsp
Primary Platform : Apple Computer Inc.
CMM Flags : Not Embedded, Independent
Device Manufacturer : none
Device Model :
Device Attributes : Reflective, Glossy, Positive, Color
Rendering Intent : Perceptual
Connection Space Illuminant : 0.9642 1 0.82491
Profile Creator : ADBE
Profile ID : 0
Profile Copyright : Copyright 2000 Adobe Systems Incorporated
Profile Description : Adobe RGB (1998)
Media White Point : 0.95045 1 1.08905
Media Black Point : 0 0 0
Red Tone Reproduction Curve : (Binary data 14 bytes, use -b option to extract)
Green Tone Reproduction Curve : (Binary data 14 bytes, use -b option to extract)
Blue Tone Reproduction Curve : (Binary data 14 bytes, use -b option to extract)
Red Matrix Column : 0.60974 0.31111 0.01947
Green Matrix Column : 0.20528 0.62567 0.06087
Blue Matrix Column : 0.14919 0.06322 0.74457
Current IPTC Digest : aefa1090a310ed6495879afe7d5caf10
Coded Character Set : UTF8
Application Record Version : 4
Time Created : 11:43:18
Digital Creation Date : 2017:06:18
Digital Creation Time : 11:43:18
XMP Toolkit : XMP Core 5.1.2
Creator Tool : Capture One 10 Macintosh
Lens : 50 mm
Flash Compensation : 0
Date Created : 2017:06:18 11:43:18
Legacy IPTC Digest : AEFA1090A310ED6495879AFE7D5CAF10
Image Width : 7360
Image Height : 4912
Encoding Process : Baseline DCT, Huffman coding
Bits Per Sample : 8
Color Components : 3
Y Cb Cr Sub Sampling : YCbCr4:4:4 (1 1)
Aperture : 4.0
Date/Time Created : 2017:06:18 11:43:18
Digital Creation Date/Time : 2017:06:18 11:43:18
GPS Altitude : 121.7 m Above Sea Level
GPS Date/Time : 2017:06:18 11:43:18Z
GPS Latitude : 51 deg 10' 57.28" N
GPS Latitude Ref : North
GPS Longitude : 0 deg 50' 32.62" W
GPS Longitude Ref : West
GPS Position : 51 deg 10' 57.28" N, 0 deg 50' 32.62" W
Image Size : 7360x4912
Megapixels : 36.2
Scale Factor To 35 mm Equivalent: 1.0
Shutter Speed : 1/160
Thumbnail Image : (Binary data 2689 bytes, use -b option to extract)
Circle Of Confusion : 0.030 mm
Field Of View : 39.6 deg
Focal Length : 50.0 mm (35 mm equivalent: 50.0 mm)
Hyperfocal Distance : 20.80 m
Light Value : 11.0

Bloody hell! What a lot of information!
You may or may not want to work your way through all of that lot, and indeed, much of it is fairly useless or meaningless. But I will draw your attention to some interesting snippets:

Look at that last bullet there, with the GPS coordinates. They are absolutely spot-on. If you were to plug that data into Google Earth you'll see exactly where we sat down for our family picnic the day we visited Bird World! Furthermore, delving deeper, it seemed that the camera was moving at 0.07 km/h and was facing 237.17 °. BUT .... please note: check your manual: GPS may not be ON by default! Certainly, on my Pentax, it has to be switched on!

Now, as I say, an awful lot of the EXIF info there is of little or no use, but I can imagine that, perhaps, scene of crime photographers might make good forensic use of some of it.

Filtering the info, saving it, using it
The EXIF you see above was obtained by executing, in a terminal window:
exiftool ALAN4918.jpg

But I could type this instead:
exiftool -aperture -shutterspeed -iso -focallength ALAN4918.jpg

In which case this is what I see:
Aperture : 4.0
Shutter Speed : 1/160
ISO : 125
Focal Length : 50.0 mm

Had I given the name of the containing folder instead of a single picture EXIFTOOL would list those details for each picture in the folder. So as I continued my EXIF journey I thought - hey, why not store useful data in a database, and then subsequently I could calculate juicy snippets like average aperture per lens, the relative ratios of lens useage, oh, it goes on! It can get really really geeky. For this sort of approach to work I needed to save the data somehow, not just show it on the screen. I discovered this EXIFTOOL syntax:
exiftool -T -directory -filename -aperture -shutterspeed -iso -focallength -datetimeoriginal >allmypics.txt /Volumes/Actinium/Pentax K-1/0167

In words: give me a tab-separated file called allmypics.txt containing directory, file name, aperture, shutter speed, date and time taken of all the pics in that folder. A bit of the result:
/Volumes/Actinium/Pentax K-1/0167 ALAN4907.PEF 2.2 1/160 125 50.0 mm 2017:06:18 11:23:12
/Volumes/Actinium/Pentax K-1/0167 ALAN4908.PEF 2.0 1/60 125 50.0 mm 2017:06:18 11:23:44
/Volumes/Actinium/Pentax K-1/0167 ALAN4909.PEF 5.6 1/200 125 50.0 mm 2017:06:18 11:24:35
/Volumes/Actinium/Pentax K-1/0167 ALAN4910.PEF 2.0 1/160 125 50.0 mm 2017:06:18 11:25:03

This all took place on my iMac. I transferred allmypics.txt to my Linux machine, and set about writing some software which would extract the values and insert them into a database.

Like the true software geek that I am I later had fun calculating that I'm most likely to use f/7.1 with my 14mm lens, and ISO 250 with my 100mm, and lots of other worthless and silly stats.

But then, I thought, why leave it at that? I worked out this syntax:
exiftool -T -directory -filename -aperture -shutterspeed -iso -focallength -datetimeoriginal -gpslatitude -gpslongitude -gpsaltitude >allmypics.txt /Volumes/Actinium/Pentax K-1/0167

And here is a bit of the result:
/Volumes/Actinium/Pentax K-1/0167 ALAN4907.PEF 2.2 1/160 125 50.0 mm 2017:06:18 11:23:12 51 deg 10' 56.93" N 0 deg 50' 28.73" W 109.1 m Above Sea Level
/Volumes/Actinium/Pentax K-1/0167 ALAN4908.PEF 2.0 1/60 125 50.0 mm 2017:06:18 11:23:44 51 deg 10' 57.01" N 0 deg 50' 28.61" W 109.9 m Above Sea Level
/Volumes/Actinium/Pentax K-1/0167 ALAN4909.PEF 5.6 1/200 125 50.0 mm 2017:06:18 11:24:35 51 deg 10' 57.15" N 0 deg 50' 28.67" W 110.1 m Above Sea Level

From little steps to something that impressed even me
Right geekwad, so you've got all those GPS coordinates and you've already established that a) they're accurate and b) you can manually type them into Google Earth (let's call it GEarth from now on) and see where you had lunch. You've amended the Linux software so it now stores coordinates and altitude too. Big deal! Loads of DB'ed data.

But what about getting a whole album's worth of coordinates into GEarth? More research!

GEarth, I already knew, could import a so-called KML file and show what it represents. I've been doing this for ages, taking the output from my quadcopters and, via quite a tortuous pathway which it's not necessary to go into, getting GEarth to show my flight. Here is the GEarth representation of one particular session. It's shown as a yellow line and the location is Greenham Common:



What you're thinking: I know. I hadn't been flying for long and wasn't very adventurous.

Irrelevant note: Greenham Common is fascinating, though most of the original US airbase has gone. A few treasures remain, one of which is the firemens' practice plane, seen lower-left in the picture above. Look it up in Google!

Google very kindly make no bones about their KML format, and the full API is published. Using what I read there, together with the example given, I amended my software to make it produce a KML.

Nervously, I opened GEarth and opened the KML. I was delighted to see exactly what I had hoped to see! This:



And here is a zoomed-in capture showig where the aforementioned ALAN4918 was shot over lunch:


Next steps: I would like to find out how to make that yellow pin a bit smaller, and perhaps vary its colours a little. I'd also like to make the font giving the picture name a bit smaller, but so far the code required is eluding me. But I'm perfectly delighted with what I've achieved so far. I think I might remove the .PEF extension from the placemarks too.

Oh, you thought I forgot didn't you - what does a KML file look like? Here is 0167.kml:
<?xml version='1.0' encoding='UTF-8'?>
<kml xmlns='http://www.opengis.net/kml/2.2'>
<Document>

<Placemark>
<name>ALAN4906.PEF</name>
<Point>
<coordinates>-0.84130277777778,51.1824722222222,109</coordinates>
</Point>
</Placemark>

<Placemark>
<name>ALAN4907.PEF</name>
<Point>
<coordinates>-0.84131388888889,51.1824805555556,109.1</coordinates>
</Point>
</Placemark>

[shortened]

</Document>
</kml>


And the very last word
In the KML above you will see

<coordinates>-0.84131388888889,51.1824805555556,109.1</coordinates>

But in the EXIFTOOL output we had:

51 deg 10' 56.90" N 0 deg 50' 28.69" W

This flummoxed me, and I had to hit the interwebs again for clues. How do I convert 51 degrees 10 minutes and 56.90 seconds into a single floating point number?

It turned out to be easy: 51 + 10/60 + 56.90/3600

But time passes, and so do many interesting minutes (OK then, hours) of exploration and imagination. I've done a hell of a lot more work on the whole GPS story, and you therefore owe it to yourself to look at the next page of this saga to see where I am with it right now. The saga continues. Page 2.