[pct-l] PCT gain/loss; notes from the actual Halfmile source

[David Lippke]

Hi, Lon alerted me regarding the PCT elevation gain/loss stats conversation
here and so I thought some direct information might help.   I'm the author
and maintainer of the Halfmile apps and I generated all the elevation data
and gain/loss numbers published for the last three annual cycles.   I spend
most of my time working on improving both the horizontal and vertical
accuracy of all our PCT data.   This effort has ballooned over the last
couple years involving more than just myself and Halfmile with the
deployment of survey grade equipment to the trail, custom GPS logging
devices, and now further generations of all that we are scrambling to
deploy for the 2014 season.

Anyway, people often wonder how we calculate elevation gain and loss
numbers along the PCT.    It turns out that this is a surprisingly complex
and technical process.    The net is that we currently generate profiles
that are much more accurate than, say, the ones that Google Earth
produces.   That said, there is still substantial room for improvement and
we are working on that.

The technical TLDR is that our point elevation values are produced by
heavily processing USGS DEM 1/3 arc second data.   Our gain / loss
calculations then operate over this set using a smoothing factor consistent
with the average error present in the DEM data and the average horizontal
"side to side" path jitter observed along side slopes.

Most will want to stop reading here.   100% tech talk follows.   You have
been warned.  :)

Going on in more detail, the process does not involve using GPS elevation
data at all.  As noted by Brick, elevation values obtained from GPS units
are much too noisy for use in gain and loss calculations -- even from the
survey grade equipment that we now use.   Instead, our raw source for
elevation data is the USGS 1/3 arc second DEM (Digital Elevation Model)
which provides average elevation values over rectangles that are
approximately 8x10 meters on a side (the "8" varies with latitude).

We start with our best filtered GPS horizontal data for each bit of
trail.    This ranges in horizontal accuracy from 1/2 meter (California
sections L through O) to 3-5 meter accuracy (e.g., WA J) to unknown
accuracy data that we hand select from the competing data sets.   Over 2014
for the 2015 update, we hope to bring the average horizontal accuracy for
the entire PCT to the vicinity of a single meter.

Then we translate all those horizontal points to the horizontal datum used
by the USGS and we fetch the DEM elevation values needed generate a grid of
525 points surrounding each subject point.   Using those 525 points, we
generate a two dimensional spline interpolation and query it at the exact
point.   This process is repeated for all 3/4 million horizontal data
points and involves almost 11 million unique 1/3 arc second DEM values.

So at this point in the process we have the best possible estimate for the
elevation of each track point and way point expressed in terms of the
NAVD88 vertical datum (quasi MSL) with the primary error sources being that
of the USGS DEM (RMSE ~2.5 meters) and that of our collected horizontal
positions.

In the next step, we reduce the horizontal point count so that all sections
stay within Garmin's 10K track point limit.   We do this by applying the
Ramer-Douglas-Peucker algorithm with a 1.9 meter sigma overall and, in more
accurate sections, a 1.25 meter sigma.    In other words, when the trail is
going along a straight road or the aquaduct, the points can be several
hundred meters apart but in tight turns they may only be a couple meters
apart.

Finally, to calculate gains and losses between points, a tally is kept from
track point to track point except that no gain or loss is recorded until a
track point is reached that has an elevation loss or gain more than 5
meters from the starting track point.   When that occurs, the algorithm
moves forward to the current end track point and picks up the counting
again.   When reporting the gain / loss to a particular way point, any
residual in the smoothing is closed out so that the numbers all "add
up".    As a side note, this creates a small numerical issue when one
decides to total our published per-section numbers over the whole trail and
compare those to the whole trail numbers in the (soon to be released) 2014
version of the Halfmile apps -- our per-section numbers are "closed out"
with respect to this 5 meter smoothing but the app just keeps rolling
through the boundaries.

Sorry for the tech talk, but I just wanted to address all the speculation
about what feeds into these calculations.    Questions are always welcome
and so is project involvement -- we have all manner of "task" available for
the inclined.  :)

All the best,
   David Lippke aka White Jeep

p.s. to contact me directly, please write directly to
lippke at gmail.comversus the .list address you see here.
Alternatively, you can find me on
Facebook or Linkedin.