Did you ever wonder why polls consistently report finding that 90+% of Americans want universal background checks preemptively-prove-your-innocence prior restraint on rights, and other such nonsensical infringements…and it rarely passes when put to a referendum? Even when it does pass, actual voting results never come within 30 points of the claim.
Blame poor polling technique. Or good technique, if your goal is to invent support for things you know don’t actually have popular support. To find this you have to go beyond news stories about a poll’s results, and look at the poll itself.
Most often, questionable polling results are attributed to asking the wrong (or right) question. You certainly see that in virtually every poll on gun control proposals.
We saw an excellent example of this when the Texas Tribune declared that a recent poll showed overwhelming support for ex parte “red flag” confiscation laws. The problem is that the poll never actually asked anything about that.
Q30. Do you support or oppose allowing courts to require a person determined to be a risk to themselves or others to temporarily surrender guns in their possession?
The courts, of course, can do that now under current laws on the books in every state. The question fails to mention the defining characteristics of new “red flag” orders: ex parte and an absence of probable cause.
Ask one question, present the results as something else.
Another area subject to abuse is the respondent selection methodology or who you’re including in the poll results. Ideally, you want a random sample statistically likely to represent the overall population, and the sample should be large enough do the job.
Most pollsters now avoid one of the selection traps: calling listed landlines. That once-useful, pre-cell phone technique now tends to select an older sample of respondents. The preferred method is random-digit dialing; you’ll get both young and old, rich and poor…a group more likely to represent the population as a whole.
Once you have someone on the phone, there are multiple techniques used to pick one respondent out of a household.
GOOD: The Kish method; asks for the adult with the most recent birthday.
GOOD: The Troldahl-Carter method; birthdate-based, but alternates things like male/female.
BAD: Self-selection; this is why online polls need to be taken with a huge grain of salt. Respondents decide to participate based on having a specific interest in the subject. There is no randomization.
BAD: Quinnipiac, noted for blowing political predictions, polls a mere 15 “regions,” heavily biased toward blue/purple states. And they not only survey the entire state of New York as a “region,” they also poll New York City as its own separate region. Most of the country is left unrepresented. I view Quinnipiac polls only for their entertainment value, although they may be useful for Democrat-specific surveys like primary candidate preferences.
HORRIBLE: “For the landline sample, interviewers requested to speak with the youngest male member of the household who is at least 18 years of age; if there was no male in the household, interviewers requested the youngest female.” This technique is becoming very common; anecdotally, I’ve been told by people who all the telephone survey calls they get now ask for the youngest person; usually the youngest female, and male if no female is available.
That automatically biases the responses, as younger voters tend to be more liberal. And, sadly, in my experience that is also the demographic least likely to understand existing firearms law. Young, ignorant socialists; what could possibly go wrong?
Perhaps a poll doesn’t disclose its methodology, or it appears reasonable. It helps to spot check claims, like the 90% background check approval that is never reached in real voting.
A Pew survey claimed to have found that 30% of adults possessed firearms. Of that group, 19% supposedly claimed to be National Rifle Association members.
The problem is that worked out to over fourteen million “NRA members,” a number more than twice as high as even the NRA has ever claimed. (Pew’s survey was presented as proving that gun owners, and NRA members specifically, support more gun control laws.)
A recent McLaughlin & Associates survey purported to find that Donald Trump has 43 million Twitter followers. Between the @realDonaldTrump and @POTUS, addresses, Trump has 93.3 million followers. Perhaps he does only have 43 million US voter followers, but I’m dubious that more than half of his followers are non-voters or foreigners following another nation’s head of state.
When all else fails, another clue about a poll is accountability. The University of New Hampshire (commonly referred to as the University of North Massachusetts, for its liberal leaning fell over completely to the left) published a survey purporting to show that 94% of New Hampshirites wanted universal background checks. The poll question did specifically ask about that, but…
The result struck many people as highly unlikely for the state’s then-demographics. No one could find anyone who admitted to participating in the survey, pro- or con-background checks.
When asked for the raw polling data, UNH refused to release it.
Nonetheless, the state’s Democrats pushed for background checks. It failed. And in the next election, annoyed voters turned the Executive Council, State Senate, and State House over to the Republicans.
If a pollster won’t show you his work, don’t trust his polls. If he will, make sure it really is what he claimed.
I like how Carl chose a girl who looks like a bored 19-yr-old college student to represent his article on people who read and discern gun polls. Not sure if it’s humorous or ironic.
In any case, I pay absolutely no attention to polls from any source, on any topic. They can be skewed by the source, or interpreted by the reader, in any direction away from the truth.
Glad we agree.
https://www.usnews.com/news/health-news/articles/2019-09-13/most-us-parents-say-vaccination-should-be-requirement-for-school-poll
So Tony… would you know anyone by the name of P G 2?
You beat me to it. P_g_2 = TonyL
Looks like TTAG has expanded its filters to include any variation of that troll’s name. I wrote this follow-up a moment ago with three variations (but not the original name), and now they also get your comment blocked. He must have really pissed off the admins.
No Toni or tony is not pg… there can be more than one of them in our forums… isn’t Toni the one who threatened to stab a nurse with a needle in her neck if she tried to give him a vaccine?
No, no and definitely NO.
“No” to whether you know P_g_2? Or “no” to the claim that you wanted to stab a nurse with a needle, lol?
Just sayin’, tho, anyone who brings up the vaxx gets a raised eyebrow. It was a long, hard fight to get “that other guy” to finally shut up and leave.
Tony/Pg, why do you haunt a gun blog specifically arguing your agenda on vaccines? You have no other drive in life. You comment on guns or politics or gear. Why don’t you start your own vaccine blog? You could even call it TheTruthAboutVaccines. I’m just saying your automatically not going to be well received here because you’re constantly thread hi jacking.
Sorry Merle, you’re barking up the wrong tree. “You comment on guns or politics or gear”…isn’t that exactly what this blog is for? You lost me.
I mistyped. I meant to say you don’t comment on any guns or politics or gear. You only comment about vaccines. Why do that on a gun blog? For example, Would it make sense for me to constantly bring up “climate change” On every article I visited here?
Sorry my posting history doesn’t sit well with you. But this is the internet, maybe if others people ‘s posts annoy you so much you should refrain from participating on online forums? Just saying.
Or maybe your just triggered because I figured you out? 😉 Pg pg pg….
Who’s triggered here? Think you have a nice imagination, and maybe projecting a bit? You seem obsessed with Pg pg pg?
Actually, I originally chose a different image. I believe Dan picked this one.
Conservatives don’t poll on guns because they know the answers already
The only gun polls are done by liberals, and therefore, are deceptive by definition
“Figures often beguile me,
particularly when I have the
arranging of them myself; in
which case the remark
attributed to Disraeli would
often apply with justice and
force: “There are three kinds of
lies: lies, damned lies and
statistics.”
Mark Twain
Polling in and of it’s self has always been a poor indicator of how society feels on a topic. Having spent some time working in the polling field. I can attest to the fact that most polls are taken with either a preconceived idea of the results or developed to achieve a preconceived result. Questions are often posed in such a way as to lead the respondent to answer a particular way. The second reason polls are ineffective in determining a overall conclusion on a subject. Is simply sample size. Most polling is done on groups of 1000 respondents or less. Making the entire premise of the poll moot. at the end of the day the only poll that truly addresses how society feels on a particular issue is an election. Polling in today’s context has become nothing more than a instrument with which the media and so called experts can argue their talking points.
Well said. Polls are an attempt to drive opinion not represent it.
Lies, lies,lies and liars.
I rather liked the 2016 election poll that had the Harpy Hag as a shoe in against Trump,only thing is she Lost. Now there is talk of her wanting to become a three time Looser,she must be a glutton for punishment that or stupid .
“No possible path for Trump to the presidency”…I’ve answered my phone to an old NRA gal “polling” me exactly once. No one has gotten through again! Anyone else EVER answer a gunpoll?!?
I a word, NO.
“…purporting to show that 94% of New Hampshirites wanted universal background checks.”
OK, so this isn’t true. But let’s make a distinction here.
There’s what the poll says and then there’s what people in the media claim the poll says. More often than not when you get a strange item like this the reality is that the poll doesn’t say what the media says it does and the pollsters, when questioned, don’t support the statements in the media (which is why they’re rarely talked to by the media).
Yes, questions matter and for the most part they’re carefully crafted. However, the people that do this professionally are very careful to tell you what conclusions can and cannot be drawn from the question/responses as the question was actually asked. The media OTOH, not so much.
For example, the poll referenced here, when we look at the actual data, has the 94% number in it twice. Once for 94% of self identifying Democrats favoring a state level AWB and once for “Strongly Favor, Issue: Read Boston Globe”.
When it comes to background checks the highest number study itself claims to find is 88% for “Background checks for felons”.
I don’t know where exactly this “94% support for background checks” thing comes from but it’s NOT from that study.
This isn’t really surprising. Media people look for a story and when they can’t find one they make one up. They also don’t understand fuck all about polling or statistics. They hop up and down about numbers that are so far inside the MoE that it’s laughable. On the flip side they ignore statistical landslides when the poll finds things the media personalities don’t like.
It’s not usually a case of “rigged polls” so much as it’s a case of the media covering the poll improperly, misunderstanding what the poll actually says or flat out lying about it.
On a second read here, I see what the media did and I see why the poll didn’t say 94% but I know how the media came to that conclusion.
So, if we add those who “strongly support” “Background checks for felons” to those who “somewhat support” we get 94%. Those who are ignorant or liars might call that “94% support”. That’s 88% for strongly support and 7% that “somewhat” support added together. It doesn’t work that way…
The reason the poll doesn’t make that claim, well one of the reasons, is that the MoE on this bad boy is 4.1%, which isn’t great to begin with (generally if you exceed 3.5% you fucked up or had a really tough situation to poll in the first place). I wouldn’t want to defend this poll in an introductory Methodology course, never mind actually use it IRL when I had to fudge my MoE that far to get a 95% confidence interval. SPSS spits out these numbers and I’m gonna run a new poll.
Now, I’m not going to get into the exact workings of MoE and recalculating something like this here, but basically take that 4.1% and double it, that’s your general at-a-glace rule. IOW, that 7% is inside the MoE because the MoE is going to calculate out to roughly 8% but just looking at it people with even basic experience know it’s not going to be less than 7.5% and 7<7.5 so it's not even worth the time to do.
Put a shorter way, that 7% isn't actually statistically significant within the context of polling statistics and the MoE of this poll.
That 7% also has to be put in the context of the "strongly support" which has a 4.1% MoE as well, meaning it might be as low as 83.9%. So you'd need to recalculate the relative MoE's of the two numbers and post those, which again, isn't worth it because you already know at a glace that with the original MoE that 7% isn't significant so you're doing a bunch of work to say "Well, we ran these calculations but within the confines of our own poll they don't mean fuck all because they're either not significant within our confidence interval OR they break that interval by dropping to less than 95% which means that they're considered garbage and at that point we kinda broke our own poll. Professional pollsters know this, we're professionals so we know it so now we're openly admitting to tomfoolery that means you can't trust anything we say".
Apologies, it was 88% and 6%, 7% was for somewhat support “Background checks including gun shows”. Long day.
Obviously 6% makes this even worse than 7%, which was closer to being statistically significant.
You sure spent a lot of time to write a lot of words that I skipped over and never read. Word of advice…summarize to capture and keep your audience.
I didn’t write it for you to read and I don’t give a fuck about the TL:DR crowd.
The truth is that TL:DR is a lazy way of saying “I’m a lazy, ignorant fuck and I like it that way”. Which is exactly how we have arrived at a lot of sad passes we find ourselves at today.
Nuance is the watchword and nuance isn’t explained in a couple of sentences.
Or, hey, you can just watch the antis use that sophistry skill of theirs to whittle away at your rights until you have none.
If you want the Spark’s Notes on the things that really matter you’re asking to get fucked and probably not smart enough to realize you’re bending over and begging for it.
no sweat, math is hard.
i hung on every word.
thanks for clearing all that up.
i have arrived at some exciting new conclusions.
@strych9,
Lol, chill dude. Just busting your balls over the (really long) length of your comments. The advice about keeping it short was genuine.
However…I *do* find your snark about “I didn’t write it for you to read” to be ironic. You didn’t spend all that time writing for us to read it?
Some if us like to learn something. It’s not like he was pasting in the same screed for the hundredth time. It was original material that required research and thought, and it is meaningful for those of us who have a little knowledge in the area.
“Those who are ignorant or liars…” So basically, the press and politicians.
Winston Churchill once said, “If you are not a liberal at twenty you have no heart. If you are not a conservative at forty you have no brain.” I skipped the liberal at twenty part.
What? Any poll we see in print is garbage. And unless certain on the air networks took the poll it is probably even less accurate. Kind of like when Barak the Anointed said that 90% of NRA members favored something. How the frig would he know anything since should a poll was never conducted.
Sample sizes seem to befuddle many critics of polling-type research.
Here’s a good, simple explanation of how that should be done: https://www.surveymonkey.com/mp/sample-size-calculator/
As already noted, the real surveys are elections.
“When asked for the raw polling data, UNH refused to release it.”
We saw that with the east Anglia climate study that supposedly ‘proved’ man-caused ‘global warming’. The infamous ‘Hockey Stick’ chart.
When questioned about the raw data they used to make their conclusions, they refused to release the data.
Flat *refused* to share the raw data. I don’t know about them, but at the college I went to, refusing to provide the data set of a science experiment would be laughed at and I would have been failed for that experiment.
And yet the climate department at Anglia refused to show their data…
Kind of off topic, but completely true. There are two issues here.
First, they massage the raw data prior to use. The claimed reason being that historical measurement methods and technology changed, and part of it is correlation with other data. The other data can be weird stuff like tree ring thickness tells them the weather was colder, but the recorded temperatures don’t indicate it, so they subtract a few degrees from the measurements from that time period.
Second, the data is then fed into a computer model that takes a bunch of factor into consideration. Which factors and the influences assigned to them matter. Plus, there could be errors in the program itself. There were reports that the program that was obtained by hackers would show a CO2 rise even when input temperatures were constant.
Going back to the issue of polling (or any study), if they don’t show the actual questions, the raw categorization of the people polled, and any adjustments made, it’s worthless. Like the historical temperature adjustments, pollsters will do things like say 40% of the last election voters were Dems, but I only got 30% Dems, so I’ll increase their answers by 33%. It might be a valid thing to do, but they should be upfront and include the original data as well.
Who answers survey callers? If I dont know you or expect your call I dont answer. And if I were to accidentally answer a call and someone was asking me political questions I’d hang up.
Now if midway asks me for to complete a survey on their customer service I’ll oblige. Haven’t encountered questions about courts or guns I’d ignore them too.
I’ll buy into these things when someone actually calls me….and my phone doesn’t show… spam?
i used to have the fam in my cell as “them.” then the kids got their own and when a friend’s phone rang and it said “scam likely,” i knew i had the new contact name for ol’ girl.
I believe ALL polls are bogus.
1. Many people don’t have landlines anymore, so hard to get a representative sample.
2. Many, myself included, refuse to answer any questions for any poll.
Liars will continue to lie. Nothing has changed.
Polls which the results of which are made public are a type of propaganda and nothing else. They are an appeal to popularity, a social proof for whatever the poller wants to push.
Hillary to win in landslide was what every poll said until about an hour after she didn’t.
My thoughts and opinions are officially expressed in polls once every two years standing in a booth. No other poll actually means anything anyway.
———————————————————————–
There is only ONE way to know when your being lied to (regardless of topic).
You have to know the truth going in.
Here is a good way to tell if the poll is bogus, look where it appears: CNN, ABC, NBC, CBS, PBS, MSNBC, NYT, Yahoo, Google News should all be considered tainted sources. Virtually all phone polling is crap at this point anyway, because most people won’t answer unless the number is recognized in their call list. This is a direct result of people getting sick of phone spam, do you answer calls you don’t recognize?
Polls have always been bull shit. They were never meant to measure public opinion, they have always been used to try to shape public opinion.
There are lies, damn lies, and polls.
Here’s another example of a bogus poll, from Fox News no less. Fox ran it about a year ago if memory serves. It asked if you favored a building a “wall” on the US border with Mexico. Now how many folks who responded were visualizing the Great Wall of China or the Wall from Game of Thrones when reading the word “wall”? Unsurprisingly, a majority were against building a “wall”. Fox could have used a more neutral word like “barrier” even in another question. They could have asked if the person knew there was already a “fence” along 700 miles of the border, but didn’t. As the article recommended, read the poll’s questions.
All polls are fixed in one way or another. I like when the author says he reads polls as a form of amusement. Whether RIGHT leaning or LEFT leaning it’s only use is to confirm numbers that the “experts” have already determined. Numbers that support their bought and payed for agenda. Then they pass it off as unbiased scientific results.
Like a couple comment-folk above, I’ve worked in this field a bit. Discerning from a sample the *actual* va. *wishful* disposition of a larger bunch of folks is hard. Even “hard” A/B tests like *this page layout* vs. *that page layout* are fraught.
The people who make their own decisions from polls, interviews, or even observed behavor (vs. self-reporting) work unbelievably hard at that. And they still don’t entirely trust what they’ve found out. Because they’re still wrong a lot, no matter how hard they try.
Comments are closed.