Have you ever listened to AM radio at night and wondered how you could hear stations from great distances away, but not hear them during the day? So have scientists and researchers, who have spent decades trying to understand what's going on above us to cause radio signals to "skip" like that at night. That led to more questions, such as why a signal sent from the same location at the same time of the day - but at a different frequency - skips farther or shorter distances? Or why signals from the same location at the same time of day and at the same frequency - but sent during a different season of the year - can behave differently? Or how in some years, even with all those conditions (location, time of day, frequency, and season) the same the signal travels great distances, but a few years later will barely make it to the border? With all of these variables at play, are we able to predict the behavior of the ionosphere and its effect on communication?
Sure we can.
But no, we really can't.
Okay, so first a quick bit of history to put these seemingly conflicting answers in context. Humans have long observed - and early on exploited for navigation - the fact that a magnetized object which is allowed move freely will point north. In 1839 a scientist named Frederick Gauss postulated that there was region of space above the earth that was electrically charged. For the next several decades a lot more postulating, theorizing and experimentation went on that lead to the development of an early form of wireless communication called spark gap. The name is very descriptive. You know how sometimes you will touch someone and you both receive a small electric shock? That's basically spark gap. When early experimenters (such as Guglielmo Marconi) figured out how to detect the spark from a distance, radio communication (using a variation of Samuel Morse's code) was born.
Among its many disadvantages was that spark gap required massive amounts of power to work over any appreciable distance, and was limited to a small range of frequencies, near today's AM band. What happened next reminds me of lining up dominoes and watching them fall against each other, because the timing could not have been better for the advancement of radio and our understanding of the ionosphere. The first domino is the passing of the Radio Act of 1912. Since wireless "state of the art" in 1912 was spark gap, which produced signals below 1200 KHz (basically the middle of today's AM radio band) the Act graciously handed experimenters and ham radio operators - what Congress blithely considered to be a useless part of the radio spectrum, above 1500 KHz.
Spark Gap transmitter at the Awanui, New Zealand station. Note the blowers in the bottom right of this photo. These high-power stations got hot. Very hot.
Hear that sound? That's another domino falling.
Just before the First World War the pioneering work with vacuum tubes by Lee DeForest, Walter Schottky, and others enabled the design of transmitters that produced a signal that was more than just a hash of static, it was a tone called Continuous Wave, or CW. Amateur experimenters who could not afford the extremely powerful transmitters and massive antennas of commercial and military spark gap stations found they could make CW contacts with much less power and smaller antennas. These experimenters also began to notice patterns in their ability to reach certain distances based on some of the variables mentioned earlier such as time of day and frequency. Leveraging that experience, hams were soon making record-setting connections between the Eastern seaboard of the United States and Europe, and the West Coast and Hawaii by radio. Now get ready for another domino to fall. In the beginning most hams were crowding close to that 1500 KHz Congressionally-mandated wall. It got very crowded down there very quickly so, leveraging the ever-improving frequency range of the new tubes, higher frequency transmitters and receivers were built.
Now a seriously important domino was about to fall because in the early 1920s, as experimenters moved up in frequency they found longer distance communication was easier at certain times of the day. As more and more data was collected (hams are notoriously good record-keepers, making this like a global lab experiment) the physicists put the pieces together and, in 1923, concluded that the ionosphere - postulated 87 years earlier by Gauss - was responsible. Now a whole line of dominoes began to fall as several things happened. First of all radio, considered to be a toy in its early days, was quickly becoming commercially viable. Hundreds of local stations were being built around the country, their owners happy to sell airtime to local hardware and feed stores, evangelists, and hawkers of patent-medicines, all of whom were happy to pay to have the ears of thousands of potential converts and customers. Listener-ship skyrocketed with many manufacturers getting into the radio receiver market, including the purveyors of cheap crystal sets, which allowed folks whose homes did not have electricity to hear those broadcasts.
No electricity? No problem with a crystal set!
The government quickly realized that this new industry required oversight by an organization dedicated to taming this new "Wild West," so in 1926 it formed the Federal Radio Commission. A year later the Radio Act of 1927 was passed which, among other actions, rescinded the Radio Act of 1912. Now fall more dominoes, as those frequencies above 1500 KHz that the government had tossed away as "useless" back in 1912 were now being allocated to commercial, amateur, military and other government services. This put thousands more technicians and experimenters up and down the frequency spectrum, working with more efficient transmitters and better performing tubes, collecting tons of data on radio reception. In 1926, building on the work of theorists and physicists such as Oliver Heaviside, Arthur Kennelly, and others physicist Robert Watson-Watt first used the term "ionosphere" to describe that place where radio signals propagate or "skip" (as it was more popularly known by excited radio listeners) in the atmosphere.
What they found was that a radio station operating at the right frequency and at the right time of day could be heard thousands of miles away - even across oceans. This was not lost on some governments in Europe, where tensions had been rising since the end of the First World War. Germany and England, two nations at the forefront of wireless research, built stations that could be heard here in America, allowing them to present their views unfiltered to the world - not unlike the way Twitter and other social media are used today. Being market-driven, it wasn't long before manufacturers of consumer radios included one or more shortwave bands in their radios. This meant that many Americans who were listening to Franklin Roosevelt's "Fireside Chats" broadcast here in the States were also able to hear Hitler, Churchill, and other foreign leaders via shortwave, thanks to the skipping of signals in the ionosphere.
Zenith Model 6-S-222 "Cube" Radio from 1936, with three radio bands
Ionospheric skip took on even greater geopolitical importance after the Second World War, as both sides in the Cold War built mammoth shortwave stations, pumping out up to a million watts of power to extol the virtues of their political system and the disadvantages of the other. Yet our understanding of the ionosphere, especially our ability to predict the behavior of signals up there, was not advancing fast enough for those tasked with the development of rockets, whether they were being designed to deliver nuclear payloads to the enemy or humans into orbit. But it wasn't just the ionosphere that was of concern. Based on observations made as far back as the early 20th century, scientists were speculating that there was a region of charged energized particles beyond the ionosphere. In January 1958 experiments aboard Explorer 1 (the first United States satellite to reach orbit) proved the existence of what became known as the Van Allen radiation belt.
Very early observations of the Van Allen belt confirmed what had been speculated by scientists, that from 93 million miles away the sun was producing a "solar wind" of electrically charged particles that pushed against the belt at various times and at varying strengths. While there was some predictability to it (such as an 11 year cycle of sunspots) the sudden appearance of solar flares were very unpredictable. So, for those of you keeping score, between the discovery of the radiation belt and the sun's impact there were dozens more variables added to an already long list (frequency, time of day, time of year, etc...) that need to plugged into any equation for ionosphere skip prediction. And what an equation it is! (Don't worry, I won't give you any more math than I can handle... which is why I am giving you none.)
At the very beginning of this piece I asked if we could predict the effect of the ionosphere on communication. My answer back then was: "Sure we can." And then I said "But no, we really can't." I hope you can see that I was not being deliberately evasive. The fact is that even today, after over 180 years of theorizing and testing and collecting data (from both terrestrial and space-based observations) physicists and mathematicians kind of throw up their hands when asked for a definitive predictive model for what's going on up there by giving us this word:
stochastic: /stəˈkastik/ adjective
Derived from the Greek, the word means "randomly determined; having a random probability distribution or pattern that may be analyzed statistically but may not be predicted precisely." What that means for us is that instead of using lined up and orderly dominoes for my ionospheric analogy I should have employed the game of Craps, because Einstein was wrong when he said God does not play dice with the universe. Not only does He play dice, but He's constantly shooting for a hard eight.
I know this from personal experience as a ham radio operator, and now that you have all the background, I want to tell you about the time I cashed in on the long odds. To start, hams are always looking to improve their chances of making contacts with other hams in far-off places (in ham parlance, we call that DX) and there are a number of online tools which provide predictive reports on ionosphere skip. Here's an example of one such report, posted on the American Radio Relay League's website, which shows what happens when all those variables we've been talking about get plugged into a tool that charts predicted activity between any two parts of the world, in this example the East Coast of the United States and Eastern Europe:
Source: American Radio Relay League
The vertical axis is frequency in MHz (for reference, an AM station at 1030 would be down at 1.3) and the horizontal axis is any 24-hour period at UTC (what used to be called Greenwich Mean Time) during the month for this chart. Now, here is why we have to swap our dominoes for dice, because the three different colored lines represent only the probability that at a given time and frequency two hams could make a connection between those two places on the planet. Red represents a 10% chance, green is a 50% chance and blue is your safe bet (kind of like using house money) with a better than 50% probability that those two hams could hear each other at that time and frequency.
The chart says that If I tuned my radio to 14 MHz at 0800, I'd have a 50/50 chance to connect with a ham in Prague. If, at the same time, I transmit at 10 MHz I'd have a better than even chance. That drops to only a one in ten chance were I to be on 18 MHz. So 18 MHz at 0800 is not a good place to try to make that contact. But we also see (by following the green line) that the chances for a contact at 18 MHz improves as it gets later in the day. Come back in six months, when the northern hemisphere is tilted differently towards the sun and the chart will be different. Come back in six and half years (during the other half of the sunspot cycle) and the chart will look very different, as well.
So thank you Gauss, Heaviside, Van Allen and others for all your pioneering work. Thank you to the folks who collect data from satellites, radio beacons, solar observatories and other sources to create this and other reports. But you know what? While they are great guides for those of us who surf the ionosphere, I have personal experience with the predictably unpredictable nature of skip and how, sometimes, even when all the charts and tables and reports say otherwise, you gotta go for the hard eight.
In 2010 the sun was at a low point in the sunspot cycle, meaning that the ionosphere was not conducive to skipping signals, especially up on the 28-MHz band. The data said that at 0200 UTC (9:00 pm EDT here in Massachusetts) there was a very low probability for making a contact with anyone more than a few miles away. After being on my radio I was ready to concede that as fact but, before I shut the radio down, decided to tune across the band just in case. Suddenly, I caught the sound of a voice, weak at first, but quickly getting stronger. As weak as his signal was, I could tell he was calling CQ - that's ham shorthand for "can anyone hear me?" Within 20 seconds it was as if he were in the room next to me.
"This is E51JD in the North Cook Islands calling CQ and listening for a call."
Wait. What? No, that's impossible. The North Cook Islands are (...type a quick lookup on Google Maps...) over 7,000 miles away. And the charts and tables say that... okay, just in case... "E51JD this is WB2HTO in Massachusetts, do you copy?"
"I do and good afternoon, WB2HTO this is E51JD on North Cook Island in the South Pacific. Name here is Jim..."
So I began a conversation with a location that the charts and tables said I should not be able to have. It was a quick exchange of basic information (signal reports, names, and a few other details) because just as his signal had been growing in strength it was now starting to fade. So we quickly signed off while we could still hear each other. As soon as we were done I could hear other stations trying to reach Jim, but whatever was happening up there in the ionosphere was changing so rapidly that I'm not sure if anyone else made a contact. I had been listening at just the right time and on just the right frequency to take advantage of what the physicists call a sporadic E opening, the "E" referring to a layer of the ionosphere that sometimes acts... stochastic.
Since, like Forrest Gump's box of chocolates, you never know what you're going to get (or how long it will last) when the E layer starts acting up like this, I moved up in frequency (to give Jim a clear space to make other contacts) flipped on the microphone and called out a "CQ" looking to see if any stations would reply. No sooner did I release the mic button that I heard "WB2HTO this is VK2HOT in Port Macquarie Australia, how copy?"
This... is... impossible. Port Macquarie Australia (another quick Google search) is 2,000 miles farther away than the North Cook Islands. And I'm talking to someone there. At night. On a band that is supposed to be dead.
The distance from Port Macquarie, Australia to Boston is over 9,000 miles
A quick exchange of basic information followed and the exchange ended while we could still hear each other. Meanwhile the band continued to shift. I only had two data points - the North Cook Islands and Eastern Australia - but it looked like the skip from Boston was moving westward. You can bet I tried calling for another contact, but after several tries it was clear that whatever magic had happened up there in the ionosphere was over. Or perhaps there just wasn't a ham in the right place at the right moment to hear me.
Several weeks later I got something in the mail from my new friend. It was a QSL, or confirmation, card. They are mostly sent electronically today, but I'm glad to have this hanging in my radio shack to remind me that the ionosphere is predictably unpredictable. Or, perhaps, unpredictably predictable.
Confirmation, or QSL card, from the contact that the numbers said should not have occurred