I managed to get the data from AR50F9911180207 digested, which was quite a success, and then processing the next chunk was a breeze. However, I made a few small changes and began looking more closely at some of the data. In particular, I examined closely the ending star positions. I have the program spit out the final positions of the stars it has identified in the output, so that I can compare these with the actual values. This will give me a good idea as to how accurate the locks are.
The results were depressing: about half of the stars had suffered from significant drift. Here’s how it happens: the star is going along just fine with a good solid lock, then encounters some bad frames or the star blinks out. This may sound incredible, but even the brightest stars are subject to disappearing occasionally. This will normally last only a few frames, but in its desperation to re-lock onto the missing star, the software might just lock onto a nearby star instead. When that happens, the constellation of stars used by the system to register the frame in the sky is slightly distorted. That distortion in turn makes it easier for another star, some time later, to jump its lock when things get noisy. Slowly, star by star, they jump to new locations, and the whole system drifts.
To address this problem, I added a new fragment of intelligence to the star-location algorithms: I weighted stars by the length of their history. You see, each star is measured relative to all the others in the set of identified stars. Thus, each identified star helps decide what new stars to add. Previously, all stars were treated equally, but now I have altered the software so that stars whose lock has lasted a long time are given greater weight in this calculation, and newly-identified stars, which could well be in error, are given little weight. This made a huge improvement in the integrity of the system, but it still wasn’t enough to prevent star drift. So I tightened the specification for how accurately a star must fit into the existing set of stars in order to be accepted. The new specification is 10 milliradians, or about 0.6º. In truth, most stars are localized to within 3 milliradians, so I should be able to do much better, but in practice, I have found that the search algorithms need a little elbow room to find stars. I am currently running a test with a 5 milliradian limit; so far it seems to be holding, but I’m not optimistic.
I have been tempted to introduce yet another improvement into the system: weight stars by their distance from the test point. Nearby stars would be given greater weight than distant stars. This would reduce the problems introduced by spherical distortion, but I am reluctant to do this because in all my tests the degree of distortion in the frame has turned out to be tiny.
A number of other possibilities remain:
1. Extend the range of star searching.
Currently, when tracing a star-flare, the algorithm looks only at the four pixels that are immediately above, below, left, and right of the current pixel. This solution would extend the search for two pixels in each of the four directions. The reason for this can be seen in the following blowup of a star:
Note the dark pixels surrounding the star: these are the problem created by the image intensifier. Each pixel in the intensifier frees up some electrons whenever photons hit it; those electrons are then used to created the intensified image of the pixel. The problem arises when the pixels in the star itself receive more photons than it has electrons to dispense. The pixel “steals” electrons from neighboring pixels. This darkens the neighboring pixels. Those darker pixels create a wall that my algorithm cannot penetrate. Remember, my algorithm steps from pixel to pixel, looking for abnormally bright pixels. It stops when it encounters a dark pixel. In other words, unless my algorithm starts inside the prison created by the dark pixels, it has little chance of discovering the star.
Allowing the algorithm to take two steps from its current location would permit it, in some cases, to step over the dark pixels and get to the star. Unfortunately, when I tried this out, it didn’t seem to help much.
2. list star flares
This would be only a diagnostic procedure. My algorithm searches for bright spots in the frame that might be stars. It might be useful to know how many such spots it finds; if it finds very few on each frame, that’s my problem, and I need only be more liberal about accepting bright spots as possible stars.
3. retain lost stars
Currently, when a star is lost from view, I throw it away. I partially compensate for this by giving a star “three strikes” -- it can disappear for two frames in a row, but if it’s still not there on the third frame, then it’s out. This addresses some of the flickering that we get with this image. While this helps, it’s not enough. The new idea is to retain a star -- but not use it -- for a longer period. In other words, we lost the star image, but we’ll continue looking for it for an extended period. The flaw with this idea is that the most common reason for losing a star is a combination of bad flickering (or glare) with aircraft motion. The frame really wanders around; stars are constantly changing their positions. This is a slow process, but if we lose the star while it’s moving, there’s no hope of recovering it.
4. keep track of overall frame motion.
This is a promising idea. First I search for all the previously-identified stars. Some will be lost, but most will be retained. Make a note of how far each star has moved since the previous frame, average together the results, and then apply that difference to any stars that have been lost. It might be possible to find a lost moving star by looking at the location where we would predict it to be, rather than where we last saw it.