Thursday, April 11, 2013

Exploring Hospital Patient Survey Data With Excel GeoFlow

As we add more OData features to FlatMerge, we are having fun discovering new ways to work with data. This post highlights 3 interesting things:
  1. The newly added FlatMerge OData $metadata support enables opening FlatMerge data directly into Excel Data Explorer. (note that FlatMerge offers minimal, but essential bits of support for $metadata currently)
  2. Some interesting data that is geocoded using Bing in Excel! Survey of Patients’ Hospital Experiences (HCAHPS) from data.gov.
  3. GeoFlow for Excel. From Microsoft: "GeoFlow lets you plot geographic and temporal data visually, analyze that data in 3D, and create interactive "tours" to share with others."
Here's how we loaded data from FlatMerge into Excel and used GeoFlow to see which hospitals are recommended by (sampled) patients, exploring using 3D maps!

First, we grabbed the text data from data.gov and changed the column names (they were way too long, like sentences). We also replaced "Not Available" to empty so calculations will be happy. Numbers were text percentages appended with %, so we removed that too.

Get the data and OData link here



Then we copied the OData link (http://flatmerge.com/Data/odata/9548f35f-7c02-4bca-a2e9-61d2b407fe92) and pasted it into the Excel Data Explorer -> From Other Sources -> From OData Feed


After clicking Apply, Excel queries the metadata about this OData feed, which just includes this one data feed (the survey data we've uploaded to FlatMerge). In FlatMerge, we provide one OData URL for each data file. So, choose the only table in the feed, which is identified by the GUID assigned by FlatMerge.  Click that and Excel queries the data to pull the top 100 rows ($top=100).


Click Done to let DataExplorer get the rest of your data. Once DataExplorer finishes downloading the rest of your data, click Insert tab -> Map (Map is the GeoFlow launcher, download it and install it here http://office.microsoft.com/en-us/download-geoflow-for-excel-FX104036784.aspx).
 
 
 
Start invstigating the data, visually, in 3D! One of the coolest things about GeoFlow is that you can make time series interactive tours of your data. This hospital survey that we grabbed is just one survey (there may be more), so no time series tour here, but the 3D navigation is still pretty cool.
 
Select "State" column for the Geography, then click Map It. GeoFlow calls Bing and geocodes the States' geographic centroids.

Then, to keep things simple , we add just one interesting data point, so select/click the PctRpt_YES_would_definitely_recommend_the_hospital column. That column appears in the Height dropdown. Select that Height again and choose "Average" to get the average respondent percentage who chose that that would definitely recommend the hospital.

We also clicked the "Chart" button to get a chart of the values. Drag around the map and have fun!

 
 
We hope this shows some interesting features of FlatMerge's OData support and how easy this functionality makes it to use with other tools like Excel and it's new toys like GeoFlow.

Monday, March 18, 2013

New Feature: Get OData Output From FlatMerge

We've added OData output through FlatMerge. Now if you have a rectangular data file you have OData too! This is an addition to the current output formats of plain XML and JSON.

We'll be doing a series of posts about the many cool things you can do with this new feature. Today, we demonstrate the usefulness of OData and FlatMerge by loading some data into Excel 2013 directly from a FlatMerge URL.

Since earthquake data is so interesting, we've grabbed the freshest earthquake data from data.gov (https://explore.data.gov/Geography-and-Environment/Worldwide-M1-Earthquakes-Past-7-Days/7tag-iwnu), and uploaded it to FlatMerge http://flatmerge.com/Data/Details/fa05d620-96fc-4e18-8d7f-e9ba0ec59c75. (You can upload data for free to FlatMerge too!)



To get OData from FlatMerge, use the proper HTTP Accept header, like this
Accept: application/atom+xml
or add the querystring parameter &format=odata

Excel automatically does this (adds the Accept header) when you choose to add OData. So, to import FlatMerge OData into Excel:

1. Go to the FlatMerge details page of the data you've uploaded or are interested in and copy the OData URL as circled in red in the above image.  Open Excel and navigate to Data->From Other Sources->From OData Feed (or PowerPivot->Manage->Get External Data->From Data Service->From OData Data Feed), and paste that URL into the Location Field.



2. Select the one and only table listing. This is the original file name that was uploaded to FlatMerge.

3. Click Next and save the connection if you'd like. Select how to view the data (we've only tested with Table) and where to put it.

4. Data is imported! (Depending on the size of your data, this may take just a second, or hours).


Please try it out and let us know how you like it.


Monday, March 4, 2013

What's in my data?

One of the coolest things about FlatMerge is that it tells you types and lengths of your data. We're relying heavily on this feature as we build in dynamic OData output (coming soon!). So, browsing around data.gov, we found some interesting raw data about Congressional campaign expenditures https://explore.data.gov/Elections/2009-2010-Candidate-Summary-File/38zs-22s9


Loading this data into FlatMerge (http://flatmerge.com/Data/Details/77e6b770-8ef6-4658-912a-a771d159f270) shows some interesting things...

The description of the data on data.gov says that ZIP Code (can_zip) is 5 digits.



FlatMerge found some ZIP codes to be 9 digits... is this ZIP+4?


Also, it seems strange that the Candidate Incumbent Challenger Open Seat (can_inc_cha_ope_sea) field indicates that it is a 10 character field, but has an explanation for some letter abbreviations.


Turns out, the explanation of the abbreviated data is not needed, because the data contains the full text.

 

So, the documentation does not exactly match what's truly in the data.

Sunday, February 24, 2013

Use Excel 2013 with FlatMerge

In Excel, click Data tab -> From Other Sources -> From XML Data Import

Select the Data query location from FlatMerge, here we open the most recent Earthquakes and choose format=xml



Excel 2013 is ready for import
Have some Excel 2013 fun!





Friday, January 11, 2013

Farmer's Market Map

We've created a Google Map of Farmer's Markets using data in FlatMerge. This post shows how FlatMerge makes it simple.



1. Grab the csv data from data.gov and upload it to FlatMerge.

2. See what's in the data. X, Y (longitude and latitude coordinates, cool!), and website are good for now.


2. Browse Google Maps API examples for some cool ways to make maps.

3. Create a lean OData query to get the fields I need.
http://flatmerge.com/Data/Query/31974ef9-5a62-420d-a6c3-945b3ab8e6d4?$select=x,y,website

Load markers on the map using the x/y coordinates from the JSON returned by FlatMerge query:
$.getJSON("http://flatmerge.com/Data/Query/31974ef9-5a62-420d-a6c3-945b3ab8e6d4?$select=x,y,website", function (data) {
for (var i = 0; i < data.length; i++) {
var market = data[i];
var myLatLng = new google.maps.LatLng(market.y, market.x); 


4. Check it out http://flatmerge.com/markets.html! View the source code to see how simple it is. (Note that the code won't work using a local .html file, it must be on a real web server or localhost.)

Mobile app and how-to post coming soon...