I was asked to ground truth the results of the population estimates for the Provincial Electoral Districts (PEDs). I had been given a paper copy of the tables with the results of the estimation procedure. The tables showed the PED names with columns of the years with the absolute and annual percent change. Since I has no know of where the PED area names referred to, I searched for and eventually found the PED boundary shapefile for ArcView(GIS) and attempted to bring in the digital spreadsheet, but there were problems with the area names and the shapefile was missing some PEDs. I regularly find that there are problems with the geography.
When I put the PED total population estimates into the GIS to compare them with the previous years estimates by CDCSDs and by LHAs, it was easy to see that the results of the PED estimation procedure were very wrong. Areas that were exactly the same for the different boundary sets had very different estimates of population change. For example, the New Westminster PED is exactly the same area as New Westminster CSD but had completely wrong numbers. Indeed all of the Lower Mainland showed very different results.
I asked Dave O’Neil and Jennifer to come and see the results. When Dave O’Neil saw how I had done the comparison (using GIS instead of the paper copy) and the results, he became agitated and critical. He said that he would never do the project this way and that there were too many colors and that I needed to redo all the maps with fewer colors by dumbing down (in geography generalization is not affectionately called dumbing down) the interval data all the way to categorical for areas with over 2% change in population totals.
I pointed out to Dave and Jennifer that if one area were changed (e.g. New Westminster) then all areas would change and therefore it is a waste of time to redo the maps until the model had been adjusted. Dave O’Neil agreed but regardless he wanted the maps redone to his specifications and a full report on a co-workers desk in the morning. Here again, there had been little to no planning. Curiously, I was not invited to be involved in the methods or the modeling for this project while everyone else in the population section was and therefore I had no idea about the time line again.
Instead of focussing on the obvious problems with the procedure that had come up with such wrong numbers for the PED estimates, Dave focused on ‘the look’ of the maps. It appears to me that he becomes agitated when fundament flaws in the logic behind the methods are exposed by myself. I worked late to change the map colors as per Dave’s’ specifications, but found that valuable information was lost. Therefore, I changed the maps back to the way I had done them in the first place so that he could see the difference. Again, a GIS is far superior to using a paper copy to examine the data. Adopting new tools and techniques may be difficult but we should look at this not as a way of displacing procedures that people have become comfortable with, but rather as a way improving efficiency and reducing error.
The next day July 21, 2005, Dave O’Neil asked where the full report was and I restated that a change in one area would result in a change for all areas. Since it was obvious that the method had produced wrong results, it would be necessary to change the areas in question (e.g. New Westminster PED) and that the exercise would have to be rerun anyway and would give different results for all the PEDs in the province.
Nonetheless, I had redone the maps as per Dave O’Neil’s specification (staying late again) and also redid the maps the way they had been done in the first place because important information had been lost using Dave O’Neil’s dumbing down method and I wanted to show him this. Dave O’Neil agreed that the dumbing down exercise was not worth it. It appears that whenever I show an improved way of doing things to Dave O’Neil and a co-worker at the same time, Dave O’Neil gets agitated and defensive; however, if I show him without a co-worker present, he is far less defensive and agitated. I requested that I be included in the methods in the future whenever possible. Sometimes it takes time to adopt new ways. I just could do without the criticism.
I also asked one of the programmers (Keith Morris) to write a simple routine in VB (it took less than 5 minutes to write the macro) that would use the GIS to automatically check for differences between the results of the estimation procedure against the estimates for other boundary sets. Therefore, instead of having a human who is not familiar with the boundaries (e.g. PEDs) check a paper copy of a table of the results of the estimation procedure, it could be done in seconds by anyone (i.e. the people involved in developing the methods and model).
I had suggested that the estimates procedure go first into a GIS and that since estimating population for different areas is a geographic exercise, that geographic software be used right off the bat. When I described the automated approach to Jennifer and explained how this would take care of the problem with the current methods, I was told that "there are always problems and that I just had to get over it," (where upon she shouted to Dave O’Neil who was near by) "right Dave". I said that I don’t agree, but rather that improvements should be considered. Indeed, the Ministries Misson Statement advocate innovation, and creative solutions.
I later found out that Jennifer had been given the lead on this project as well as on the projections and the school district estimates and projections.
Also, there is a major flaw in the method used by Dave, Jennifer, and Frank. The method used to ground truth the population estimates for PEDs created by Dave O’Neil using a conversion program required that I check for possible errors by indicating estimates that are too high and too low. Later I found out that Frank then took my suggestions and arbitrarily took people out of areas I thought could be high and put them into areas that I thought were low. Frank then passed me the changes for approval. I spoke to Dave O’Neil about this method saying that it is less than scientific and improvements should be made as I do not think that it is right for me to sign off on a project that is done this way. The automated process that I had written is far more reliable.