Google refine tutotial
Download as PPTX, PDF5 likes3,531 views
This tutorial provides an overview of using Google Refine for data cleansing tasks. It demonstrates how to import data, perform faceting to analyze columns, remove redundancies, perform numeric faceting, cluster similar text values, use expressions to transform data, augment data by linking to external sources, and export cleaned data. Steps shown include importing disaster data, analyzing columns through faceting, merging similar column values, calculating new fields with expressions, linking to external sources to enrich locations data, and exporting the refined data.
More Related Content
Viewers also liked (13)
Similar to Google refine tutotial (20)
Recently uploaded (20)
![Fast Screen Recorder v2.1.0.11 Crack Updated [April-2025]](https://cdn.slidesharecdn.com/ss_thumbnails/dataanalysisforbusiness-250322061148-eeff8a831-250401123246-f36be9ca-250401141614-87b01766-thumbnail.jpg?width=560&fit=bounds)
Google refine tutotial
- 1. Vinod Gupta School of Management, IIT Kharagpur Google Refine Tutorial April, 08 2012 Sathishwaran.R - 10BM60079 Vijaya Prabhu - 10BM60097 This Tutorial was created using Google Refine Version 2.5 on a Windows 7 platform
- 2. Data Cleansing • Data cleansing is identifying the wrong or inaccurate records in the data set and making appropriate corrections to the records. • It involves identifying incomplete, inaccurate, and incorrect parts of data and then either replacing them with correct data or deleting the incorrect data • Data cleansing results in data which is consistent with the other standard data and is useful for performing various analysis • The error in the data could be due to data entry error by the user, failure during transmission of data or improper data definitions. 2
- 3. Need for Data Cleansing • Incorrect or inaccurate data may lead to false conclusions and can cause investments to be misdirected in finance. • Also government needs accurate data on population and census for directing the funds to the deserving areas. • Many organizations tap into customer information. If the data is not accurate, for eg. If the address is not accurate then the business runs the risk of send wrong information, thus losing customers. 3
- 4. Challenges Data Cleansing • Loss of Information: In many cases the record may be incomplete, hence the whole record may require to be deleted which leads to loss of information. It could become costly if huge number of data is deleted. • Maintenance of Data: Once the data is cleansed then any change in the data specification needs to affect only the new values. Hence data management solutions should be designed in such a way that the process of data entry and retrieval are altered to provide correct data. • Data cleansing is an iterative process which needs significant work in exploration and corrction of entries. 4
- 5. About Google Refine • Google Refine is a powerful tool that can be effectively used for data cleansing. • It helps in working with raw data, cleaning it up, transforming from one format to other, encompassing it with web services and linking it to databases. • It is very easy to use and has a web interface. • It is freely available and works well with any browser. • Google Refine is a desktop application and it runs a small web server on your system and we need to point our browser to the server to use refine. 5
- 6. Getting Started - Installation 1. Download the zip file (appropriate Windows, Mac, Linux versions) from the link http://code.google.com/p/google- refine/wiki/Downloads?tm=2 2. Uncompress the files from the zip file. 3. Run the “google-refine.exe” file. 4. A command window opens and Google refine runs taking the user to the home page in the default browser. 6
- 8. Importing Data • Google Refine supports TSV, CSV, Excel (.xls and .xlsx), JSON, XML, and Google data document formats. • Once imported the data is in Google Refine’s own data format. • We have used TSV data on Disasters worldwide from 1900-2008 available from http://www.infochimps.com/datasets/disaster s-worldwide-from-1900-2008 for the tutorial. 8
- 9. Importing Data 9
- 10. Importing Data 10
- 11. Data Uploaded Creating Project 11
- 12. Creating Project Project Created 12
- 13. Faceting • Faceting is about seeing the big picture and filtering based on rows to work on data you want to change in bulk. • We can create a facet for a column to get the details about that column and then we can filter to a subset of rows with a constraint. • We can perform text facet, Numeric facet, timeline facet and scatterplot facet. Also various customized facets can be designed. 13
- 14. Faceting 14
- 15. Faceting The Column Type has 18 unique options 15
- 16. Removing Redundancy Even though they are of same type, shows as different options due to case 16
- 17. Removing Redundancy 17
- 18. Removing Redundancy 18
- 19. Removing Redundancy 19
- 20. Removing Redundancy Reduced to 15 unique options 20
- 21. Numeric Faceting 21
- 22. Numeric Faceting Highly clustered towards low values 22
- 23. Numeric Faceting 23
- 24. Numeric Faceting 24
- 25. Numeric Faceting Cost column is blank and has no value 25
- 26. Numeric Faceting Calamities with low cost 26
- 27. Numeric Faceting Calamities with high cost 27
- 28. Clustering • Clustering is used to merge choices which look similar. 28
- 29. Clustering 29
- 30. Clustering Data Merged 30
- 31. Using Expressions • Expressions are used to transform existing data to create new data 31
- 32. Using Expressions 32
- 33. Using Expressions 33
- 34. Data Augmentation • Reconciliation option in Google refine allows data to be linked to web pages. Suppose we want details on the country where the calamity has struck we can perform the following steps 34
- 35. Reconciliation 35
- 36. Reconciliation 36
- 37. Reconciliation 37
- 38. Reconciliation 38
- 39. Reconciliation 39
- 40. Data Enrichment 40
- 41. Data Enrichment 41
- 42. Data Enrichment 42
- 43. Data Enrichment 43
- 44. Export 44
- 45. How to Use Twitter Data Step 1 Step 2 45
- 46. Step 3 46
- 47. Step 4 Step 5 47
- 48. Step 6 48
- 49. Step 7 Step 8 49
- 50. Output 50
- 51. Friends Events using Facebook data 51
- 52. Friends Events using Facebook data 52
- 53. Friends Events using Facebook data 53
- 54. Friends Events using Facebook data 54
- 55. Friends Events using Facebook data 55
- 56. Friends Events using Facebook data 56
- 57. Friends Events using Facebook data 57
- 58. Friends Events using Facebook data 58
- 59. Friends Events using Facebook data 59
- 60. Friends Events using Facebook data 60
- 61. Friends Events using Facebook data • After splitting the cell using separator },{ 61
- 62. Friends Events using Facebook data 62
- 63. Friends Events using Facebook data • After updating for other columns and rearranging it we get the events as 63
- 64. Thank You 64