[Skimmertalk] Beta test of spot quality filtering for RBN spots

[Pete Smith N4ZR]

It's been a busy and rewarding couple of weeks.  Sometimes, I'm quite 
overwhelmed by the willingness of hams to invest large amounts of time 
and intelligence in advancing our hobby, not for any personal gain but 
simply because they care and enjoy what they are doing.

This is one such case. The AR Cluster V6 Telnet server at the RBN has 
just been replaced with a beta version that provides quality scores for 
each RBN spot, as well as filters enabling users to apply those scores 
to limiting the spots they receive. Note that if you don't set any of 
the new filters, the node will continue to function as it always has, 
except for the addition of a validity code as part of the comment on 
each RBN spot.  All existing filters will continue to work as before.The 
full story is at 
<http://www.ab5k.net/ArcDocsVer6/UserManual/ArcDx_CT1BOH.htm> 
<http://www.ab5k.net/ArcDocsVer6/UserManual/ArcDx_CT1BOH.htm>.

Special thanks to CT1BOH, who did extensive analysis of RBN data and 
developed the innovative algorithms being used, and to AB5K, for the 
hard work of coding and testing the filters for incorporation in this 
new version of his cluster software.

While users will see the new tags in spots coming from the RBN ARC6 
Telnet server, in order to use the filters the node they are connected 
to must be running the beta server software as well.Users can test with 
the RBN node at arcluster.reversebeacon.net, port 7000.Node sysops who 
want to try the beta are encouraged to contact AB5K.

These are the validity codes:

? - Not yet verified.  The first and/or second identical spot of the 
same station, including erroneous spots of callers
V - Valid, meaning that at least three identical spots (call and 
frequency) have been posted by Skimmers worldwide
Q - QSY?, meaning that it is the first and/or second spot of a station 
on a new frequency, where spots of that station were previously 
verified. Sometimes, this will be a legitimate QSY, but the tag may also 
indicate an I/Q image of a good spot.
B - Bust, based on whether a new spot is enough like ones already seen 
and tagged Valid, except for a difference in the callsign.This is based 
on a really clever applied math concept called the Levenshtein distance. 
Google for more info.
. - Unique, meaning that there are only one or two stations currently 
spotting stations in a given country.  Will often change to V if more 
stations spot it, but in the meantime you won't miss that P5 because 
only one RBN station heard it.

You can filter so that you get only Valid spots, or so that you can 
block all busted spots, or so that you get no Q spots until they are 
verified (so I/Q images will not come through).  You can even tell the 
cluster node to let through spots with a "." tag, so you don't miss the 
really rare one who is only spotted by one or two stations.  Full info 
and examples at the URL above.

I do not assume that this is the last word on improving RBN spot 
quality.  Beta means beta.  Please experiment, see what you think, and 
let me know.  One particularly fruitful line of inquiry would be to 
compare the arrival time of unfiltered spots and spots that have been 
judged Valid, to note instances when an apparently legitimate spot 
doesn't get through or is judged to be a bust, or when a bust is not 
caught.

In this connection, in testing we have noticed that sometimes a Busted 
call will be mistakenly judged Valid, because there are more busts than 
good spots in a time window.  Often, this seems to be due to spacing 
errors (RN4ZR for N4ZR), frequent omission of portable designators by 
ops (N4ZR instead of P5/N4ZR), and PTT cutting off the first dit (E4ZR 
for N4ZR)

Please send your comments/reports to me, and I'll see to their onward 
distribution. Do not send them to this reflector!



73, Pete N4ZR
Check out the Reverse Beacon Network at
http://reversebeacon.net,
blog at reversebeacon.blogspot.com.
For spots, please go to your favorite
ARC V6 or VE7CC DX cluster node.