MAPSTORY LOCAL: How-To

ObTAin DATA
Find and explore your local assessor's web mapping platform.

This guide walks you through the various parts of creating or adding to a MapStory Local project. You may also want to take a look at the overview of MapStory Local, as well as the corresponding videos. All of this is a work in progress and it will be expanded on as MapStory Local develops.

Do a web search for [Your city/county] assessor, such as Ames Iowa Assessor, and navigate to the assessors site. Chances are, in the top few results, you will see a link to an assessor in your city or the county youre in. Youll want to find the web mapping platform, and explore the area youre in. It works just like any web mapping platform, where you can zoom in and out and pan and click on things. Confirm that they have a Year Built category for properties. This is the key information that you need.

Obtain the Data.

Look around on the site and see if there is a place you can download the data. In the best case scenario, it will be on the site, where you can download it, and the data will have the year built columns in it. This is rare however, and you will probably need to call the assessor or Geographic Information System (GIS) coordinator at the city. Youll probably want to tell them the purpose of the project, and what youll be using the data for. Make sure that you make it clear to them in the beginning that you are asking for the data, which will be posted online, openly. At the least, you should ask for the shapefiles for all the properties, all the buildings, all the roads and the railroads, and the city limits. For the properties shapefile, you should ask and make sure that the file they give you lists the year built for each property. Also, ask if they have any historic data they may have some, you never know. In our case, they happened to have the city limits after every annexation since the 1970s. In some cases, they may give you a link to a place where you can download it or simply send you a file in an email, but in many places, you may have to go through some hurdles. For one, they may be afraid that the data is going to be posted publicly due to privacy and integrity, and in some cases they may even claim that there should be a charge for the data that is public. But as long as it does not have anything confidential according to your state law, and does not require any processing that is significant, beyond selecting and exporting the necessary and unrestricted data into a shapefile and emailing it to you, it is likely unlawful for them to restrict or charge for the data. It is worth the effort though be persistent and patient and work with people, and be intelligent about who to talk to to in order to get the data. You may have to work your way up the chain and in the worst case, talk to the officials with the highest authority, such as your City Councilman, or even ask officials at your state who work with local officials to obtain the data. In the case of Ames, we first asked the County GIS coordinator, who made us fill out a form and allowed us to have the data without there being a charge, because it was being used by a nonprofit organization. But when we asked the City for more data, they just sent it to us in an email quickly without any agreement, as they understood that there was no problem, and had the wisdom to understand the benefits of providing the data openly. For some other non-GIS historical data that is located at the county, we are having to work with the State of Iowa ombudsman office to obtain some documents.

mapstory.org | @mapstory | #mapstory

Obtain Maps
This is essential to MapStory Local, but for the majority of the mapping and effort, it will come after obtaining GIS data. For all the places in the world where GIS data is already available, well over 90% of the effort that goes into MapStory Local will be in less then 20% of what will be mapped everything that has existed before what exists now, all the buildings, properties, pathways, roads, railroads and features that changed or were built and removed. For this data, storytellers will have to take old maps and old information, and overlay them into a GIS platform and draw over it. For this, you will need to obtain every image of a physical map that exists of the place you are mapping, scan them if they are not already, and add them in a file. Eventually, this will be done in MapStory itself, and there will be a georeferencing or rubbersheeting tool in the site itself. When that is the case, the site will be a place where people put all maps, provided that they are not in copyright. If they are in copyright, that poses a problem, as you cannot publish them openly online. You could perhaps use it on your own computer and upload the data to MapStory, and the MapStory community will probably have to reference the corresponding shapefile instead of an image, though that is not ideal. To obtain maps, just do a google search online of [Your city/county] maps, such as Ames Iowa maps. You may also want to search historical maps. It is very possible that there will be a place where people are already doing this for your town in the case of Ames, one would come across the Ames Historical Society. In fact, even if you find images of maps, a local historical society is the most valuable resource. They are full of passionate, dedicated people that collect records. Another good place will be the your municipal or county government they often have old maps as well. You should not only call the government offices, but make sure you talk to each department, as each department does not always communicate or know what kind of information other departments have so call the planning, public works, assessor, inspections, and various other departments to see what they have. In the case of Ames, the City had maps that only some people knew about, and the Ames Historical Society did not even know about it. If youre lucky, a lot of maps are already scanned, but it is likely that you will find maps that are still not scanned. For these, if they are larger than a scanner, they should be scanned with a large scanner. Scanning a map in pieces does not actually work well, as the pieces do not align when you piece them together in an image processing program. These may be hard to come by, but ask people at your historical society, library and city where you can find a scanner that scans maps. In addition to complete maps, many cities or counties will have a recorder, which store all of the individual records that they use to track property boundaries and various other things. There is a good chance that you will be able to find a drawing and description, with coordinates of every property that has been created and subdivided, going back several decades. In the case of Ames, the recorder is at the county level, where there is a copy of every record going back to the founding of the county. It is less likely that these historical documents will be digitized, but that is changing rapidly. In the case of Ames, there are an estimated 100,000 pages of property deed records, much of which are not digitized, and the ones that are have been difficult to obtain, due to confidentiality issues, the formats and medium the documents are stored, and the willingness of public officials to do the work required to obtain and deliver them. Once we have all the documents digitized, we can have a complete picture of how Ames and the county was settled by those other than Native Americans from the time the first farms were established and subdivided and sold off. Though it may be rare, in some places this may be easily obtained and can simply be dumped online. Either way, as far as human effort, that will likely be the easiest part most of the effort will be in georeferencing each property parcel and drawing over them, one by one.

STArT MApping
Organize the data
You may want to start mapping right away, but it is important that you establish good, simple organization. Fortunately, this is not that hard. Dont think that you can skip this and worry about it later things can quickly get complicated, and if you do not store things right, it will become convoluted for even you, who created all of it in the first place. The way that storytellers would organize data would be similar, and this is one approach: Create a folder for each type: Parcels, Buildings, Roads, Railroads, Phonebooks, etc. Add the files to them, and duplicate them. If the originals are not already in a zip format, zip them so that they cannot be changed. Any time you reuse the original layer, duplicate it and unpack it. Create a folder for maps, and label the maps in a standardized format: [Year_Place], such as 1902_Ames_ Iowa_USA.

Take a look at the Data

Take a look at the shapefiles you received in a desktop GIS application like ArcMap or QGIS, where you can see them visualized, and make sure there are no problems opening the files. Take a look at the data table or the database file (*.dbf ) and see it theres a column that lists the year built dates, as well as a column that lists the dates when the properties were sold. Also take a look at the buildings layer and see if theres a year built date there. For each of the following layers, it should take only a few minutes to add see as a fully working mapstory, not including the time it will take your internet connection to upload the data. Go to mapstory.org, and navigate to the place where you upload shapefiles. There will be instructions on the site to do this under How-To, which is simple and straightforward. All you do is drag and drop the shapefile and upload it, point to the column that lists the time attributes, and then have it process.

Parcels
In your parcels layer, its possible that they may have multiple sale dates listed for each parcel if that is the case, youll want to take the earliest date. Why? Thats the earliest date that the property existed in its current shape. Before that, it could have been part of a larger property that was subdivided. Go ahead and map this layer and point to column that lists the earliest date the parcel was sold, again, using the instructions in the How-To section at mapstory.org.

Buildings
If there is a year built column, then that will be helpful and will save you a step. But it is likely that the year built dates will not be listed, because assessment is for a property, both the land and the buildings on it, and buildings are simply extra shapes. So, if thats the case, youll have to do a join, where you combine two tables using a common column as a reference. In this case, you will need to join the building code to the corresponding parcel code in the property layer. You can probably do that with the address or some feature in the table. But you may need to do a spatial join. You can find instructions online to do that in the program you are using.

Roads
It is very unlikely that there will be dates attached for the roads, and you will have to do the initial work in generating a layer based on the proximity to other data, like buildings and parcels. With the help of some GIS developers at Iowa State University, a tool was built that generates a roads layer using features. In the case of Ames, we used the buildings shapefile that was created using a join, as mentioned above, and for every street segment, the earliest

date among the adjacent parcels was added. So, for example, if there are 10 buildings along Main Street, between two adjacent intersections, between 2nd and 3rd Streets, and the oldest building was built in 1905, then the date will be added to that street segment from the oldest building, and it will appear in 1905. It takes a computer about an hour to generate it fully, and the result is a layer that roughly approximates how the roads were built in a city. The first roads are more sporadic, but becomes increasingly accurate as you move the timeline forward. Its important that the right features are matched to the right road segments for example, if a building on the corner of Main Street and 2nd Street is equidistant to both, then something like the address should be used to generate the layer. But you should check the data to see how often the address and location correspond. Sometimes youll have buildings or parcels where the address is actually far away. We encountered this with the ISU campus and government buildings, which had a mailing address far away. Furthermore, believe it or not, for those of you who have not been outside an address paradigm like the US, they actually dont name streets in many countries, and addresses are based on buildings and blocks. This is the case in countries like Japan and India, which are more established societies rooted in a less mobile geography. This was made clear again with Iowa States campus, where things are pedestrian-based. Standing on the campus, you would guess that an address would say a building and office number, and not a street number, because its people oriented and not based on vehicular mobility. In these cases, roads are simply whats in between the blocks and buildings, while in most cases in the US and similar environments, buildings and blocks are what are between the roads. In developments that were created for modern vehicles, it is more likely that the destination was the result of the road, while in more pedestrian geographies, the road was the result of the destination or created concurrently. Interesting, huh? So, the roads tool was built to have two options if the features have addresses that mention the street number that it accesses, then the first option allows you want to mention those streets. The other takes a street and does it purely based on proximity its closest proximity, not using the addresses. For the buildings for which the addresses were not in proximity, we separated those buildings and generated roads separately using them. This is not ideal, as it should all be done in one layer, and will be one of the first improvements that will be made to the tool next. Another option that may provide accuracy would be the access points to buildings. This is becoming common, as this is often for safety, like police or fire, which might use GPS devices as aids. This would be especially useful in places where they dont have street addresses or for areas where the addresses are not for the adjacent streets. But for places with street addresses, it may more accurately reflect the first street, so it might be useful as a separate column that is, unless in some places the address was changed to another adjacent street.

People and Businesses

Scanning phonebooks will take a large amount of manual labor, and it is good to estimate the amount of time it would take. Phonebooks today are very dense, and can be estimated at 1 page per 1000 people, and 45 secs/page pair, while earlier directories were smaller and less dense, and could be 5 times that ratio. In the case of Ames, the number of pages only roughly doubled while the population increased from a few thousand around 1920 to 60,000 in 2012. All in all, we would estimate the time to be 50 hours of work, including 40 hours of scanning. You may think that this is not worth the effort, but it is common for people to do things like this in historical communities everywhere. People do all sorts of data entry and manual copying and preservation in historical societies. If you approached them, they might be willing to gather a crew or do it slowly over several weeks, like a phonebook a day. We are experimenting with different techniques right now, using digital cameras and even automated book scanners, to find the easiest way of doing this reliably. As this is built over the coming years, this will be useful by itself for historical societies and records, but also for understanding all sorts of social and economic trends, and tracking people with genealogical records. While most of the infrastructure that has existed probably exists now, even if you were to digitize and scan and map every directory, it would amount to a small fraction of the 108 billion people who have ever lived. Anything further than directories would have to use genealogical records, which would also have to mention where the person lived.

The process to map everything in the phonebook is simple, and starts with using a good program for optical character recognition (OCR) to convert the image to text. We have not demonstrated this with any open source OCR programs, but it is likely that you will really want to use a professional program, until free programs are at the same level. We have used ABBYY Finereader, which does an outstanding job in converting over 80% of a phonebook accurately. If you do not wish to purchase it, you can send the images to someone in the MapStory Local community who will OCR the images for you. You should not publish the phonebook pages online for anyone to download and only send it to someone you are in direct contact with it is the information, not the images, that are not open to distribute publicly, unless they are out of copyright. There are some features that you may benefit from if you have your own copy of a professional OCR software if you click on the text for example, it will take you to the part of the image that was read, and you can see how to correct it. We have yet to figure out a good manual workflow for correcting mistakes, but in the meantime, you can have the images available and an initial database file from which a shapefile can be created. Once a digital text is created, simply run the code that corresponds with the phonebook. So far we have only added a Polk City Directory from 1921, and the code is available in a free repository. The MapStory Local community will publish and refine the code over time for each phonebook publication format. Eventually, a storyteller would scan a phonebook and find the corresponding publisher and date, verify if the format corresponds using a representative layout, and then apply the right code. Once the code is added to a database or spreadsheet, the database is used to generate dots using an address locator. This is a nearly instantaneous process, using current addresses. Of course street names and numbering changes, and new address locators will have to be generated. It is likely that with both the roads and phonebook data, storytellers would also build common historic address locators. For anything you add to a mapstory, you will also need to remove anything that is private information, especially current residents and phone numbers. There are standard ways that governments and industries have for dealing with privacy, and we are choosing and developing the protocols that work to protect people. Typically private data is not openly available, or visible but not searchable. We are developing ways of dealing with data, ranging from requiring storytellers to remove current residents or designate them as current so they are not searchable. It will take time to develop policy and protocol, so in the meantime you must in fact remove residents names and phone numbers entirely, and only show the addresses, while for businesses, you can add whatever information you wish from phonebooks. Until the protocols are developed, have the original copy saved that contains all the data.

Annotations, Information
You should add annotations and information to a mapstory. The instructions for annotations are all online, under the How-To section. Eventually, features will also have information that people can add, but this is far away. In the meantime, for any given layer, you can create a separate column that says information, and add it to the shapefile before you upload it.

Georeferencing or Rubbersheeting
This is where you take all the maps you gathered, and overlay them into the geography you are intending. You can find instructions online for doing this in various programs. MapStory will have this in the site itself eventually, along with good instructions, but in the meantime, you will have to use documentation found elsewhere. Simply take each map and a layer, probably your parcels and roads layers to map it to, and reference points. Many maps are drawn to precisely to scale, but for others you will have to learn practice through trial and error and get it as close as you can.

EDiTing DATA
Eventually, editing can be done in MapStory itself, and all edits will be tracked and can be reverted easily with the click of a button. In the meantime, there will be a workflow that will resemble the workflow that will eventually be done within the site. We are developing this workflow, so this will be an early overview that may have some shortcomings. For any layer, you will probably do some manual editing, but the first layer that would require the most would be the roads layer. We will use that as an example, but the instructions are the same for any layer: Prepare and add everything: Add the generated roads layer Lock the original year built column that was generated by the roads tool; if possible, make the column read-only, such that any changes would require someone copying the file and editing a new one. Regardless of whether that capability is not available, make sure there is an original zipped duplicate of the generated roads shapefile. Create a separate column for manually entered dates. - two columns that are Start Date and End Date Add a column for labeling changes. This will refer to the layer or map that the manually entered date is based on. Create a Notes column. This is for notes where the reason something changed was not by referring to nonmapped resource or some other reasoning. Add any rubbersheeted maps. This can of course be done as you rubbersheet maps, on the fly.

Start editing: Make all the roads one color, say black. If using year, say 1902, create color scheme, where, say red <= 1902. All of the road segments before 1902 will appear red, while the other roads will remain black. See where data is insufficient, and select the road segments that need to be changed. Open up the table that shows the selected road segments, and sort sequentially. Change values that are greater than 1902 to 1902. Label edits depending on what is being used, eg a map labeled 1902_Ames_Iowa_USA, or the generated roads layer or column Generated_Roads. All of the roads should appear red. In ArcGIS and perhaps other programs, there are time sliders that can be used to see how your data has changed as you have changed it, which you should use to see how things changed. If there is no time slider, that will make things more difficult, but it can still be seen year by year through color coding.

Eventually, all of this will be in MapStory, and it will show all the data at once, with and automatically color the years according to where your time slider will be, and you can select features and change the years with the click of a button, along with the appropriate labels that are pulled from a drop-down or automatically applied.

mapstory.org | @mapstory | #mapstory