Hi,
I agree with Jerry. I spent a few weeks trying to use FT8 to obtain
antenna radiation patterns and I discovered how the SNR is computed:
it's totally flawed. Basically WSJTX computes the number in two steps:
the first one estimates how strong the adjacent frequency bins are with
respect to the bin of interest. Then this value is corrected by
estimating the baseline noise in a 2.5kHz bandwidth.
The bad is that in this way the program is trying to unify a QRM concept
(how strong adjacent noise is) with band noise (due to QRN or whatever
wide band there is).
I found out the problem because using WSPR SNR estimations lead to
antenna patterns that are very similar to NEC simulated antennas but FT8
data produced WORSE patterns for the BEST antennas. This is due to the
fact that best antennas receive far more signals so WSJTX gives a worse SNR.
I tried to talk with Joe Taylor about this, but he said that this method
is what he thinks it is best for a general use, if I have a better one,
just go and modify the code yourself.
I have a feeling that this way of computing the SNR was chosen because
it makes you think, as Jerry pointed out, that FT8 can magically decode
signals that are absolutely not hearable. That's highly dubious.
What FT8 does achieve is an extreme overlapping of signals over 2.5kHz,
but at the expense of not really being able to tell the SNR...
If you want to dig deeper in the FT8 implementation have a look at AB1HL
Robert Morris' FT8 Python implementation that is heavily commented and
very very educational: https://github.com/rtmrtmrtmrtm/basicft8
He also wrote a full code/decode high performance software (not so easy
to understand...): https://github.com/rtmrtmrtmrtm/weakmon
Hope it helps!
73,
marco / IS0KYB
On 01/08/19 15:17, K4SAV wrote:
I get the feeling that I must be the only person that has ever tested
FT-8 to the extreme to see what it can do. It seems that everyone else
just assumes it will do what the published information says. It will
not. Below is a summary of my testing.
First I did a bunch of testing to see if I could figure out how the
software determined the S/N number. I measured the strength of a
station calling CQ and subtracted that from the S meter reading for
the 2500 Hz USB band and that resulted in a number very close to what
FT8 reports. I repeated the test multiple times.
Official documentation says FT8 can decode signals 24 dB below the
noise floor. That's an interesting comment because FT8 has no way of
determining the level of the noise floor. Even during the off period
when no one is transmitting it can't determine the noise floor because
the noise floor is not represented in the audio signal. The receiver
ACG brings the noise back up when no one is transmitting. Actually
for FT8 the amount of audio is greater in the off period than it is
when stations are transmitting. The only way to measure the noise
floor is by making the measurement in the RF world, which FT8 can't do.
Actually what the program does is count everything in the 2500 Hz
bandwidth as noise. (That is comprised mainly of strong signals, not
noise.) It is limited to decoding signals 24 dB below that level.
From that info you can guess that if you narrow the bandwidth to
eliminate most of the strong signals FT8 will decode weaker signals.
Yes that works. Verified it myself and others have also found this to
be true. You guys that like to operate FT-8 should take note of that
comment.
I thought maybe it could decode stuff below the noise floor if all the
signals were also below the noise floor, so I tested for that. I
found out that the program poops out long before the noise floor is
reached. With my tests where I decreased the gain of signals from the
NE (just after sunset in the NE) by pointing my antenna west and
increasing the gain such that the biggest signals were 15 dB above the
noise floor, FT8 just about quits. There were probably 50 or more
stations on the band and it was making one decode about every 5 or 10
minutes.
FT8 has a deep search mode where it uses stored data to make guesses
at call signs some times. I tested that too. Before I started WSJTX
I set up a poor signal to noise ratio test. Started WSJTX in the
normal mode and it did very poorly. Then I turned on deep search,
increased the S/N and let the program look at the band for a little
while. Then I went back to the poor S/N condition without turning
FT-8 off and turned on deep search and it made a lot more decodes.
Nearly all of those decodes were reported at -24 dB. I think those
were guesses and it just assigns -24 dB for guesses.
I was fooled by the results of a test I did when adding enough noise
to the audio to cover up the signals and FT8 continued to decode the
signals. However I had previously had deep search on and it had
already memorized the band when I did that. It was just guessing that
the same station is on the same frequency as previously.
I keep hearing reports from people that claim they are getting decodes
without hearing anything in the audio. I set up conditions where that
should have happened, but it never decoded anything.
In summary. it appears that on an almost dead band, CW (with narrow
bandwidth) has about a 15 dB advantage at decoding weak signals. On a
very crowded band if FT-8 is using a 2500 Hz bandwidth, CW has a huge
(many dBs) advantage over FT-8 because FT-8 can only decode about 24
dB below whatever your S meter is reading. At 2500 Hz bandwidth in
the FT-8 band on 160 my radio usually hangs at 20 to 30 dB over S9.
I was using a TS-990S receiver with no audio processing or noise
limiting or blanking.
If anyone else has run similar tests. I would love to hear about it.
Jerry, K4SAV
_________________
Searchable Archives: http://www.contesting.com/_topband - Topband
Reflector
_________________
Searchable Archives: http://www.contesting.com/_topband - Topband Reflector
|