blankblank blank


Archive for the 'calibration' Category

Convert Scanned/Digital Map Images Into Offline Android Format With MAPC2MAPC – Part II

Continued from Part I, Part II is a walkthrough of converting a calibrated map or aerial image into an Android-app-friendly format.

There are a number of ways to get calibrated maps and aerial photos that MAPC2MAPC can convert to Android-app friendly format.

– GIS/mapping programs can usually export maps into “raster”/digital formats that MAPC2MAPC can use. These can also display GIS vector data like shapefiles, then convert it into a compatible rasterized format. This is useful since there currently aren’t any good Android apps that can directly display such vector data.

– Some US state and government agencies provide easy online access to calibrated scanned maps and aerial imagery. Data for the US is particularly easy to find:

For this walkthrough, I’m going to download an aerial photograph and topographic map for my neighborhood from the USGS Seamless Server Site.  I won’t go through the exact details of the process (there’s a tutorial on using the Seamless Server available at that site). But I will point out that when you select a data type for downloading, the Seamless Server will conveniently tell you the projection and datum the data is in, which you’ll need for MAPC2MAPC. Here, both the aerial imagery and topographic map are in the UTM projection, Zone 12N; however, the aerial imagery is in the NAD83 datum, while the topo map is in the NAD27 datum:

seamlessinfo

If you miss it there, or can’t find it, the downloaded zip file will include a .prj text file that has this information, and which can be opened in any text editor:

sszip

ssprj

The two files you will need for use with MAPC2MAPC are the image file (with the .tif file extension) and the “worldfile” with coordinate calibration data (.tfw file extension). In the MAPC2MAPC program, choose File => Open Calibration. In the lower-right corner will be a drop-down menu that lets you select the type of calibration file you’re using from the ones that MAPC2MAPC supports; here, I’ll choose “World files”, then select the .tfw file.

selecttfw

Because worldfiles do not contain map projection/datum info, MAPC2MAPC will prompt me for projection/datum info:

wfparams

From the Seamless Server or PRJ file, I know that the image is in the UTM coordinate system, Zone 12; the numbers in this listing box represent the UTM zones. If it were in a different coordinate system, I could scroll in the box and see if it was available at the bottom; MAPC2MAPC currently supports about 22 additional coordinate systems along with UTM. If I didn’t find it, I’d have to recalibrate the map manually using one of MAPC2MAPC’s built-in utilities, or a separate program (see post I for more info on datums, projections and map calibration). The default datum for MAPC2MAPC is WGS84; if the map is in a different datum (NAD83 in this case), I click the “Not WGS84” button, and choose the correct datum from the next window:

datum

If successful, you should get a screen that looks something like this:

2010-09-14_172052

Now the map image can easily be converted into the Android-app-friendly format. In this case, I’m converting it to TrekBuddy format, so File => Write map as Trekbuddy files converts the original calibrated map image into the TrekBuddy-compatible format.. You’ll find the generated map files in a subfolder located in the same directory as the original image file, with the same name as the image plus “_tiles” appended. The other Android-friendly format MAPC2MAPC supports is the “Mobile Atlas” format; you’ll find that export function in the File menu as well.

The mapset and tiles will be in the uncompressed TrekBuddy format, as MAPC2MAPC doesn’t currently support the compressed tared TrekBuddy format. However, there’s a free Java utility called jtbtar that can convert the uncompressed mapset into a smaller compressed one. Run jtbtar, select the folder with the map tiles, click on “Pack Map”:

jtbtar

And jtbtar will create a compressed .tar file and .tmi file containing the mapset data.

For this example, I didn’t actually notice any substantial space savings by going through this process. However, copying a large compressed file from your computer to your Android unit takes far less time than copying many individual tile files. The process of copying the data over to your Android unit is the next topic on this blog; there are several options, each with advantages and disadvantages.




Convert Scanned/Digital Map Images Into Offline Android Format With MAPC2MAPC – Part I

The previous two posts (one,two) have been on Mobile Atlas Creator (MOBAC), a program that can download map data from online services, and convert it into offline formats compatible with some Android map apps. But MOBAC can’t process your own scanned maps, or digital maps / aerial photographs you’ve downloaded off the web.To do that, there’s MAPC2MAPC, a free image conversion program (Windows only) that can turn stand-alone digital map images into several Android-app-compatible formats. Note: The latest version of MAPC2MAPC is no longer free; you might still be able to find the older free version on the web if you search a bit. But to fully understand how to use it, you’ll  need to know some basics about how digital map images are calibrated, i.e. how do you know which geographic coordinates are associated with each individual pixel element in the image.

Broadly speaking, digital map images can come in one of two formats:

Uncalibrated. Just a basic digital image with no information about which geographic coordinates go with which pixel.

Calibrated (aka “georeferenced” or “registered”). A digital image that comes with basic geographic coordinate calibration data, either embedded in the image file itself, or in a separate file, often plain text,  that can be loaded in with the image to tell the program what location each pixel is at.

For both kinds of maps, you’ll also need to know two more pieces of information that describe how the maps was drawn:

– The map’s “projection”. Maps are flat representations of the earth’s curved surface, and the projection describes geometrically both how they are flattened (in a manner usually designed to minimize distortions from the flattening process) and also the coordinate system that assigns a set of numbers to every point. Examples of projections include Universal Transverse Mercator (UTM) (which uses meters for coordinate units), Lambert Conformal Conic (also meters), and equirectangular/Plate Carree (which uses meters).

– The map’s “datum”. A datum is a set of equations that describes both the curvature of the Earth, and also the zero-zero reference point for coordinate systems. As technology has improved, the accuracy of the datum has also improved, which means that positions given in older datums may be a long distance away from the same position in a modern datum. Examples of datums include WGS84, NAD27, and ED50.

Uncalibrated maps will, by definition, lack embedded information about the projection and datum used to draw the map. MAPC2MAPC has a calibration utility that lets you calibrate the map for a limited number of projections (and a larger number of datums); this will also assign geographic coordinates to every pixel. If your map was created with one of those projections, you should use that one to calibrate your map; you will commonly find the projection and datum printed somewhere on the map, often at the bottom near the map title and scale. If the projection is not defined for your uncalibrated map, or it’s drawn in an oddball calibration, using/choosing the wrong projection can result in a distorted and miscalibrated map. MAPC2MAPC includes utilities for this that requires more calibration points, but can “warp” the map into an acceptable shape. One of those utilities lets you select points in Google Earth, and use those for the calibration process. Check the readme.txt file that comes with the MAPC2MAPC program for more info on these functions. There are also a fair number of free programs that can geographically calibrate an uncalibrated image (“georeferencing”); you can find a list of them here.

For many uncalibrated map images, though, the easiest way to calibrate the map for use in MAPC2MAPC might be to create a Google Earth image overlay file in KMZ format, then load it into the program. MAPC2MAPC accepts these KMZ files as input, and if created successfully, they’re already fully calibrated in the equirectangular/Plate Carree projection, WGS84 datum. You will likely find that most maps can be calibrated reasonably well using this process, although not all. Check out this tutorial by Garmin on the process for creating a KMZ image overlay (note: unlike images destined for Garmin units, it doesn’t have to be in JPG format, and it can be up to 100 million pixels in size). You might also check out this YouTube video tutorial and associated linked tutorials on the right of the video’s page.

For calibrated maps, the geographic coordinate data will always be included somehow in a calibration file. For some types of calibrated maps, the projection/datum information will be included in the calibration file, while for others, you’ll have to specify it. If full projection/datum data is needed for any calibration file, it will be requested by the program. Types of calibrated map files accepted by MAPC2MAPC include:

  • World files (.tfw, pgw, jgw); these always require projection and datum information
  • OziExplorer .map files; these usually have projection/datum information embedded within them
  • Google Earth overlay files (KMZ); these are implicitly in a known projection/datum
  • … and a bunch of less common ones(e.g. gux, gmi, jpr, etc). Not fully familiar with these formats, but I believe some of them have full calibration/projection/datum data, while others only have calibration data.

The MAPC2MAPC website has PDF documentation of how to load a calibration file for a USGS topo map with a tfw world file obtained from a site like LibreMap; by following this, you’ll get a feeling for how to load similar calibration files.

Tomorrow: Some sources for calibrated maps and aerial imagery, and a walkthrough on downloading and converting a calibrated aerial photo into an Android-app-ready map with MAPC2MAPC.




Compass Calibration And Reference Issues In Android Phones

Many Android phones (maybe even most?) come with a built-in magnetometer that lets them measure the Earth’s magnetic field strength and direction, and determine both magnetic and true directions for the phone. I thought I’d take a look at some of the basic issues associated with its use.

First off, the magnetometer needs to be re-calibrated on a regular basis:

  • The magnetometer readings can “drift” with time
  • Exposure to a strong magnetic field can alter its characteristics
  • Every battery has a unique magnetic signature, so replacing the battery can require re-calibration

compasscal

Figure 1: The compass calibration procedure can be accessed from the Settings menu, Location & Security => Calibration Compass. It’s kind of an annoying process requiring you to rotate the Android unit back and forth along the three possible axes of rotation until you get a message indicating that calibration is complete. On my unit, there’s a demo animation that shows you how to do this; one frame of this is the picture at left. Some apps (like Snaptic’s Compass) offer a simplified calibration procedure that involves a simple figure-8 movement of the unit; I’ve found that this procedure often doesn’t fully calibrate my Droid X compass, so I always do the standard full calibration.

I’ve found one quick and easy way to check my unit’s compass calibration. It requires that you have a regular old-fashioned magnetic compass available; doesn’t have to be an expensive one, just good enough to tell you clearly which direction is magnetic north. Next, start up the Snaptic Compass, and set it to display magnetic north. In a location away from any large hunks of iron-containing metal, which can shift magnetic fields, orient both the Android phone and the magnetic compass so that they point towards magnetic north (Figure 2):

uncal

Even a cheap compass will give you an accurate heading towards magnetic north, so if the phone and compass disagree, as they do above, it’s the phone that’s wrong. BTW, I deliberately miscalibrated the phone’s compass by doing the calibration procedure next to a large hunk of magnetic metal, my car; when you do a calibration, you should do it as far away from any magnets or magnetic metals as possible (e.g. iron, steel, nickel).

You need to be careful to leave some distance between the phone and the compass when you do this test; otherwise, the magnetic compass needle can affect the phone’s reading: (Figure 3)

adjacent

After re-calibrating the phone, and repeating the test above (Figure 4):

cal

Unfortunately, the screen contrast on the phone sucks in this photo, so you barely see the Snaptic Compass dial. But hopefully you can see that, unlike the first pic above, both compass and phone are oriented in the same direction when both indicate north.

When I use a compass app on my Android, I choose one that lets me set either true or magnetic north, and always stick with true north. True north is always the direction towards the North Pole, while magnetic north points towards the magnetic north pole, a direction offset from true north by the magnetic declination (see yesterday’s post for more on this). But some apps have no setting for true or magnetic north, and won’t necessarily tell you which kind of direction they’re using for north. For example, Smart Measure has a built-in heading meter, and it displays the magnetic direction instead of the true direction; you’d have to add the “magnetic declination” to that heading to get the true direction. And at least one app I’ve run across has a setting that lets you choose between true and magnetic north, but always displays magnetic north. The difference between the two direction angles varies depending on where you are, but is often greater than 10 degrees.

I’d like to see all app writers either use only true north, or give the option between true and magnetic north,. But until that happens, here’s a simple way to check which kind of direction an app is using:

scompasstrue

Figure 5: Install the Snaptic Compass app on your Android unit, set it to show true north in the Settings menu, then rotate the unit until it’s pointed directly at true north. Don’t move it once it’s oriented in the true north directio

magcompass

Figure 6: Start up the app in question. If the app uses true north, then the direction indicator should show north, the same as the Snaptic Compass. In this case, though, the needle deviates slightly to the east of true north, indicating that it’s displaying magnetic north. The amount of the deviation should be the magnetic declination for your current location, about 11 degrees in this case.