Saturday, March 03, 2007

The Surveys Usefulness is Dubious



The following is a comment posted in reaction to this blog topic about the forthcoming $10,000 city survey:
Leucadia!: $10,000 Warm and Fuzzy Waste of Funds

JP's suggested survey design is what is called a convenience sample. If you are looking to get very accurate estimates of what the whole population thinks, a convenience sample can be problematic and misleading. This is especially likely if your convenience sample picks up people who are a poor representation of the whole population. Setting up a booth in Leucadia is likely to produce different results than if you sampled in new Encinitas. People in Leucadia are keenly aware of how behind the City is in providing adequate infrastructure.

Another poster who snipped at [Mary] Fleener alludes to the gold standard in sample surveys. This is the simple random sample. If you sample randomly, your estimates will be unbiased and your margin of errors will be meaningful (if you calculate them correctly).

If the City's population of interest is all adults in the City, the City's survey does NOT meet the criteria to be described as a simple random sample. In a simple random sample all adults would have the same chance of being sampled. The City's survey has MANY nuances that move it away from being a simple random sample. This only matters if the bias is large enough to matter.

For the most part, it doesn't matter much to me if the estimates from the survey are mildly or moderately biased. Why? Even if we did a census of the entire population and I had the true values (not estimates that you get from a sample survey) I don't see much utility from having this information. Even if they were valuable, we need a council who are going to act on what they find.

When voting 4-0 to spend the $10k on this survey there was no discussion on how the council adjusted or acted on the previous results. The survey's usefulness is dubious and its lack of use clear.

Here are some of the questions included in the survey:

1. Is Encinitas a good place to live? My view: regardless of the "sentiment" shouldn't we be constantly struggling to make it an even better place to be?

2. Are you satisfied with fire protection services? Fortunately, only a small percentage of people have needed to call upon the fire department and even that small sub population has little means to base an evaluation of the service. Just because lots of people say they are satisfied with the fire department doesn't mean they should be satisfied.

3. Are you satisfied with trash services? Well, trash collection is pretty straight forward and something everyone has some experience with . In the last survey only 71% said they were very satisfied. The council thinks we have very good service and I don't think they are going to revisit the trash contract. Think about this too, something is way wrong with trash service long before a large proportion of the population have complaints. The city is better off spending resources letting citizens know how to inform the city of grievances, making a record of those grievances (and remediation) are archived, and ensuring nothing is structurally wrong with the service. The way I see it is that things may need to be changed even if only a small proportion of the population are affected/aware of a problem.

4. Is the City going in the right direction? Should peoples' responses be weighted the same regardless of whether or not they have been following the city activities closely or even read the paper? Most people can't name a single council person.

5. If there were one thing the City of Encinitas could do to improve your quality of life, what would that be? There were some interesting responses. Each of the following were responses from a single respondent: 1) "Keep Leucadia Funky" 2) Make city a car-free zone, 3) Get a library. Only one out of 300 said library! The library was one of the city's biggest expenditures last year (taking us deeper in debt). I would conclude that the results of this survey item are not guiding the council actions.

Stop paying to have it asked?

The top two responses to this question: 1) Stop/control growth / development & reducing / managing traffic / congestion. These two got 59% of all the responses.

Let's not waste any more money on surveys that are not useful and/or used.

Kevin C is president of the Encinitas Taxpayers Association and is a senior statistician at the School of Medicine at UCSD. He is also a pretty good surfer and his wife is the nicest person in the history of human beings.

18 comments:

  1. Thanks for bringing this post forward, JP, and thanks, Kevin for the great comments.

    I was disappointed in the City voting unanimously, but with Teresa Barth absent, to sponsor this study, another PR moment which will be used as election time approaches next year, I'm sure.

    Here are my answers:

    1. Yes, the weather and proximity to the ocean make this a good place to live. However actions of past City Councils have led to a decrease in my quality of life re traffic congestion, over-development, and loss of individual personal and property rights, increased fees and taxes.

    2. I'm not satisfied with fire protection because I think we did not need the 10 story ladder truck, and we don't need three new fire McMansions. We should phase any new fire stations in slowly, and use a green standard in building, also keeping costs down to no more than $150.00 per sq. ft.

    3. I am not satisfied with trash services because they were forced to put that illegal clean water tax, $5.00 per month increase, then took it off and raised our base fee for pick-up, instead, by about $5.00 per billing period. Seems like quid pro quo.

    4. It's too early to tell if the City is going in the right direction. I was disappointed in the decision not to put the most qualified candidate on the Planning Commission, Lisa S. with a doctorate in public policy planning, someone who is concerned about quality of life. I was disappointed with Council's vote to fund this "warm fuzzy" study, to raise the height limits, overall, at San Dieguito Academy. I was disappointed to see that City Council had closed sessions on Feb. 14 and Feb. 28, still under-reporting, and with Teresa Barth on a pre-planned trip on 2/28.

    5. Show respect for the electorate and our quality of life by considering the impact of new development, particularly higher density development in areas that were before open space. Traffic and water needs must be considered and mitigated. No more negative impact declarations.

    ReplyDelete
  2. I like the survey. I just wish the council would listen and prioritize their actions by the results.

    1. People live here for the beach and Weather. Council should "improve" (open to interpretation) the beach areas. I think N. Coast HW101 Streetscape and moonlight beach masterplan improvements would be money well spent.

    2. People are most concerned with upzoning and traffic. Council should not upzone and should leave upzoning to a vote of the public. Council should also manage traffic instead of encouraging cut through commuter traffic from Carlsbad and San Marcos .

    3. I would like a question on the survey to read, "do you want the hall property to focused on being sports complex with many lighted ballfields or more of a local community park with more openspace and walking/running trails a few unlit ball fields a skate board and dog park?".

    ReplyDelete
  3. The biggest problem is this is a company in Oregon who Jerome got the city to pay $10,000 to to call and ask 300 people basically yes and no questions - a few people I talked to said when they tried to answer in a sentence, they were cut off and asked for yes or no. Are you happy with the way the city is headed, yes or no. Do you like the conditions of your parks, yes or no. Are you satisfied with your trash collection, yes or no.
    Give me a break. This is the 3 boys on the council trying to justify their "good job." STOP using our tax dollars to do this!!!!!!!!!!!!!!!!!!!!!!!!!

    ReplyDelete
  4. My questions are:
    1. Why can't city staff do this task? What are they payed for?
    2. Why not hire a Encinitas based firm to do the survey? Or a California firm?
    3. Why doesn't the council just listen to the people of Encinitas? Why do they need a middleman to filter public opinion?
    4. Isn't there better ways to spend $10,000 in city lacking sidewalks, adequate parkland and open space?

    ReplyDelete
  5. start citizen referendums to get things done.

    I love the below part of an article on Solana Beach zoning regulations going to the voters on Tuesday. I hope the citizens could do a similar referendum- I bet it would past. Why waite?- Get busy LTC, Cardiff TC and Olivenhain TC. Talk about doing something good for Encinitas.

    Solana Beach's 'mansionization' issue now in voters' hands



    "The more I've studied it and see where the money comes from, (it) makes me really concerned for Solana Beach," Golich said.

    Golich noted that voters indirectly expressed their feelings on neighborhood preservation during last November's City Council election. Two candidates who supported the restrictions ---- incumbent Tom Campbell and challenger Mike Nichols ---- won.

    He added that in neighboring Del Mar, where property values are among the highest in the county, zoning codes provide for smaller homes.

    Based on his own observations, Golich said, "I don't see any reduction of property values in the city of Del Mar."

    If Prop. A passes, Zito said, property owners in the affected areas would still be entitled to build larger homes than they would on similarly sized lots in Del Mar.

    That punches a hole in the opponents' position that Prop. A would erode property values, he said.

    http://www.nctimes.com/articles/2007/03/04/news/coastal/2_12_563_3_07.txt

    ReplyDelete
  6. The nicest communities are where they have the strictest zoning regulations. Del Mar, La Jolla, Laguna Beach, Pebble Beach, Rancho Santa Fe. LTC this could be your one huge contribution to Encinitas. Your number 1 priority should be to get a resident initiative pasted on a similar matter to protect the future of Encinitas. You would all be heroes.

    We all know without the Citizens initiative it will never happen with the likes of developer sponsored councilmembers like Jerome Stocks, Jim Bond, and Dan Dalager.

    Do it before the next election show you can get a lot of press and show Jerome and Jim are completely bought off by the Developers “crowd” including the Building Industry Association (Patterson)

    ReplyDelete
  7. Don't restrict my property rights anymore than they already are. Zoning is great in this city.

    If you want an initiative make it that only a vote of people can change zoning. Don't let the city council have that power. Give the power to the people.

    Then the deveopers will have to get to all of us and not just their buddies on the city council.

    ReplyDelete
  8. The nicest communities are where they have the strictest zoning regulations. Del Mar, La Jolla, Laguna Beach, Pebble Beach, Rancho Santa Fe. LTC this could be your one huge contribution to Encinitas. Your number 1 priority should be to get a resident initiative pasted on a similar matter to protect the future of Encinitas. You would all be heroes.

    We all know without the Citizens initiative it will never happen with the likes of developer sponsored councilmembers like Jerome Stocks, Jim Bond, and Dan Dalager.

    Do it before the next election show you can get a lot of press and show Jerome and Jim are completely bought off by the Developers “crowd” including the Building Industry Association (Patterson)

    Check it out- Those were the days my friends- when only a hand full of folks lived in the farming communities of north county. Now its a million yuppies zooming around in there SUVs, drinking Starbucks and talking (not listening)on their cell phones- telling the poor sucker on the other end of the phone how great they are- "SHIT! gotta go, I just hit a pedestrian on HW101. Ring ya later.”

    Memory Lane: Highway 101 had its day as the way

    http://www.signonsandiego.com/news/
    northcounty/20070304-9999-
    lz1mc4theway.html

    ReplyDelete
  9. "Property value is determined by location - the character and the attractiveness of the community," says David Zito of Save Old Solana, a group supporting Proposition A.

    ReplyDelete
  10. If the City wants a TRUE random samle a Likert type survey is a better way to go. In this type of survey a person is given a choice to respond to Strongly Agree, Agree, Neutral, Disagree and Strongly Disagree. A yes or no answer is not good enough for the answers the City says it wants. Also the phone calls should be between perhaps 4-8 PM giving a more equal chance of getting all of the citizens. I sent this idea on to the Council but haven't heard a respnse.

    ReplyDelete
  11. Looks like there is advertising on this site now!

    ReplyDelete
  12. JP puts a lot of time into this blog. We should click through the ads every once in a while and support JP in that way. We could send donations instead.

    The ads are fine with me, as long as they stay mellow.

    ReplyDelete
  13. Good for JP. Its the American way. I hope he make a ton of cash, so he can support quality of life council candidates to offset the heavily funded development donations.

    You don't have to click on them.

    ReplyDelete
  14. I'm not sure, but I think now that Google has bought out Blogspot or Blogger, those ads are only there to help bring Google some money for providing the blog service, the framework.

    I doubt if JP makes anything. I didn't even see them at first. Yes, just don't click on them unless you are interested in what they offer, of course.

    ReplyDelete
  15. I allowed for various responses and even had some free response question!

    ReplyDelete
  16. Regarding the ads, I think I get 5 cents a click or something. I'm trying it out for a few weeks. If the ads start to irritate me I will remove them. I do feel guilty for using a free service even if Google is worth 50 billion dollars.

    ReplyDelete
  17. Don't feel guilty, JP. You are doing a great job, more of a public servant, in my eyes, than most people on govt. payroll.

    ReplyDelete
  18. 59 people (of 300) responded in the top two categories for what could the City do to improve the quality of life. Fifty-nine was incorrectly reported as the percentage. K.C.

    ReplyDelete

Thank you for posting on the Leucadia Blog.
There is nothing more powerful on this Earth than an anonymous opinion on the Internet.
Have at it!!!