SlideShare a Scribd company logo
1 of 17
Download to read offline
November 2011

Scratching your own itch
Joey Echeverria | Solutions Architect
joey@cloudera.com
@fwiffo
About Joey

    •  Solutions Architect
    •  6 months
    •  3+ years




2
                      ©2011 Cloudera, Inc. All Rights Reserved.
Why Apache Sqoop (incubating)?

    •  Data center integration
    •  Extensible
      –  Sources
      –  Destinations
    •  Company specific
    •  Open source
      –  Yay!



3
                        ©2011 Cloudera, Inc. All Rights Reserved.
Getting the code

    •  Apache SVN
      –  https://svn.apache.org/repos/asf/incubator/
         sqoop/
    •  Apache GIT
      –  http://git.apache.org/sqoop.git
    •  GitHub
      –  https://github.com/apache/sqoop.git



4
                       ©2011 Cloudera, Inc. All Rights Reserved.
Fork you




5
               ©2011 Cloudera, Inc. All Rights Reserved.
Building the code

    •  Requirements
      –  Ant 1.7.1
      –  JDK 1.6
      –  Maven
         •  SQOOP-306
    •  Targets
      –  jar
      –  test
      –  checkstyle


6
                        ©2011 Cloudera, Inc. All Rights Reserved.
Checkstyle




7
                 ©2011 Cloudera, Inc. All Rights Reserved.
IDE




8
          ©2011 Cloudera, Inc. All Rights Reserved.
Testing

    •  JUnit
    •  By hand
      –  ant package
      –  cd build/sqoop-1.5-incubating-SNAPSHOT
      –  bin/sqoop




9
                     ©2011 Cloudera, Inc. All Rights Reserved.
Contributing back: Why?

 •    Next release
 •    Lower long-term cost
 •    For the community!
 •    It’s easier than you think!




10
                      ©2011 Cloudera, Inc. All Rights Reserved.
Fears

 •  People won’t want my patch
     –  Rare
     –  Small projects
 •  Only I need this feature
     –  That’s what everyone says…
 •  Too much process!
     –  A little time now, a massive headache later



11
                         ©2011 Cloudera, Inc. All Rights Reserved.
Sqoop vs. HDFS

 •  Small project
     –  Less contentious
     –  Easier to grok the code
     –  Easy deployment
 •  Immediate feedback
 •  Great place to start




12
                      ©2011 Cloudera, Inc. All Rights Reserved.
Contributing back: How?

 •  File a JIRA
     –  https://issues.apache.org/jira/browse/SQOOP
 •  Post your patch on the issue
 •  File a review request
     –  https://reviews.apache.org/groups/Sqoop/




13
                     ©2011 Cloudera, Inc. All Rights Reserved.
Posting patches




14
              ©2011 Cloudera, Inc. All Rights Reserved.
The review process

 •    Upload your patch
 •    Describe your changes
 •    Describe your testing
 •    Wait for feedback
 •    Repeat




15
                   ©2011 Cloudera, Inc. All Rights Reserved.
What else?

 •  Subscribe to the mailing list
     –  sqoop-dev@incubator.apache.org
 •  Look for open issues
     –  https://issues.apache.org/jira/browse/SQOOP
 •  Get hacking




16
                     ©2011 Cloudera, Inc. All Rights Reserved.
Questions?




17
              ©2011 Cloudera, Inc. All Rights Reserved.

More Related Content

What's hot

Integrating Security into DevOps
Integrating Security into DevOpsIntegrating Security into DevOps
Integrating Security into DevOps
CloudPassage
 
Puppet Camp Charlotte 2015: Managing middleware with Puppet
Puppet Camp Charlotte 2015: Managing middleware with PuppetPuppet Camp Charlotte 2015: Managing middleware with Puppet
Puppet Camp Charlotte 2015: Managing middleware with Puppet
Puppet
 

What's hot (20)

AP Setup Using Oxalis
AP Setup Using OxalisAP Setup Using Oxalis
AP Setup Using Oxalis
 
Introduction to Infrastructure as Code & Automation / Introduction to Chef
Introduction to Infrastructure as Code & Automation / Introduction to ChefIntroduction to Infrastructure as Code & Automation / Introduction to Chef
Introduction to Infrastructure as Code & Automation / Introduction to Chef
 
Cloud Platforms for Java
Cloud Platforms for JavaCloud Platforms for Java
Cloud Platforms for Java
 
The Evolution of Glance API: On the Way From v1 to v3
The Evolution of Glance API: On the Way From v1 to v3The Evolution of Glance API: On the Way From v1 to v3
The Evolution of Glance API: On the Way From v1 to v3
 
DevOp with Me!
DevOp with Me!DevOp with Me!
DevOp with Me!
 
Spinnaker Microsrvices
Spinnaker MicrosrvicesSpinnaker Microsrvices
Spinnaker Microsrvices
 
Puppet Camp Melbourne Nov 2014 - A Build Engineering Team’s Journey of Infras...
Puppet Camp Melbourne Nov 2014 - A Build Engineering Team’s Journey of Infras...Puppet Camp Melbourne Nov 2014 - A Build Engineering Team’s Journey of Infras...
Puppet Camp Melbourne Nov 2014 - A Build Engineering Team’s Journey of Infras...
 
Support Automation Central Build
Support Automation Central BuildSupport Automation Central Build
Support Automation Central Build
 
Greenfields tech decisions
Greenfields tech decisionsGreenfields tech decisions
Greenfields tech decisions
 
Best practices deploying Sitecore to Microsoft Azure
Best practices deploying Sitecore to Microsoft AzureBest practices deploying Sitecore to Microsoft Azure
Best practices deploying Sitecore to Microsoft Azure
 
OSGi Community Event 2010 - Eclipse Virgo Update
OSGi Community Event 2010 - Eclipse Virgo UpdateOSGi Community Event 2010 - Eclipse Virgo Update
OSGi Community Event 2010 - Eclipse Virgo Update
 
Avoiding integration hell
Avoiding integration hellAvoiding integration hell
Avoiding integration hell
 
Integrating Security into DevOps
Integrating Security into DevOpsIntegrating Security into DevOps
Integrating Security into DevOps
 
Puppet Camp Charlotte 2015: Managing middleware with Puppet
Puppet Camp Charlotte 2015: Managing middleware with PuppetPuppet Camp Charlotte 2015: Managing middleware with Puppet
Puppet Camp Charlotte 2015: Managing middleware with Puppet
 
Continuous Delivery
Continuous DeliveryContinuous Delivery
Continuous Delivery
 
Security as Code: DOES15
Security as Code: DOES15Security as Code: DOES15
Security as Code: DOES15
 
Webinar: OpenStack Best Practices for Production
Webinar: OpenStack Best Practices for ProductionWebinar: OpenStack Best Practices for Production
Webinar: OpenStack Best Practices for Production
 
A Day in the Life of a Cross-platform, DevOps-enabled Team
A Day in the Life of a Cross-platform, DevOps-enabled TeamA Day in the Life of a Cross-platform, DevOps-enabled Team
A Day in the Life of a Cross-platform, DevOps-enabled Team
 
Deliver on DevOps with Puppet Application Orchestration Webinar 11/19/15
Deliver on DevOps with Puppet Application Orchestration Webinar 11/19/15Deliver on DevOps with Puppet Application Orchestration Webinar 11/19/15
Deliver on DevOps with Puppet Application Orchestration Webinar 11/19/15
 
ApacheCon NA 2015 - Gabriele Columbro - Is Open Source the right model in the...
ApacheCon NA 2015 - Gabriele Columbro - Is Open Source the right model in the...ApacheCon NA 2015 - Gabriele Columbro - Is Open Source the right model in the...
ApacheCon NA 2015 - Gabriele Columbro - Is Open Source the right model in the...
 

Viewers also liked (7)

UnicastWS vol.1
UnicastWS vol.1UnicastWS vol.1
UnicastWS vol.1
 
рынок WP август 2012
рынок WP август 2012рынок WP август 2012
рынок WP август 2012
 
プレゼンテーション02(5分バージョン)
プレゼンテーション02(5分バージョン)プレゼンテーション02(5分バージョン)
プレゼンテーション02(5分バージョン)
 
Sistema Operativo Slax Pendrive
Sistema Operativo Slax PendriveSistema Operativo Slax Pendrive
Sistema Operativo Slax Pendrive
 
Nagoya
NagoyaNagoya
Nagoya
 
SteamVRさわってみた
SteamVRさわってみたSteamVRさわってみた
SteamVRさわってみた
 
応用情報技術者第一回ゼミ
応用情報技術者第一回ゼミ応用情報技術者第一回ゼミ
応用情報技術者第一回ゼミ
 

Similar to Scratching your own itch

August Webinar - Water Cooler Talks: A Look into a Developer's Workbench
August Webinar - Water Cooler Talks: A Look into a Developer's WorkbenchAugust Webinar - Water Cooler Talks: A Look into a Developer's Workbench
August Webinar - Water Cooler Talks: A Look into a Developer's Workbench
Howard Greenberg
 
Opening opensource : The Jenkins Way
Opening opensource : The Jenkins WayOpening opensource : The Jenkins Way
Opening opensource : The Jenkins Way
Nicolas De Loof
 

Similar to Scratching your own itch (20)

Java / Opening Open Source the Jenkins Way - Nicolas de Loof, CloudBees
Java / Opening Open Source the Jenkins Way - Nicolas de Loof, CloudBeesJava / Opening Open Source the Jenkins Way - Nicolas de Loof, CloudBees
Java / Opening Open Source the Jenkins Way - Nicolas de Loof, CloudBees
 
Continuous Integration Fundamentals: Build Automation - OFM Canberra October ...
Continuous Integration Fundamentals: Build Automation - OFM Canberra October ...Continuous Integration Fundamentals: Build Automation - OFM Canberra October ...
Continuous Integration Fundamentals: Build Automation - OFM Canberra October ...
 
OpenStack Enabling DevOps
OpenStack Enabling DevOpsOpenStack Enabling DevOps
OpenStack Enabling DevOps
 
Journée DevOps : De l'intégration continue au déploiement continu avec Jenkins
Journée DevOps : De l'intégration continue au déploiement continu avec JenkinsJournée DevOps : De l'intégration continue au déploiement continu avec Jenkins
Journée DevOps : De l'intégration continue au déploiement continu avec Jenkins
 
Webinar: OpenStack Accelerates Software Development
Webinar: OpenStack Accelerates Software DevelopmentWebinar: OpenStack Accelerates Software Development
Webinar: OpenStack Accelerates Software Development
 
Java Community and Overview Track - March 2016
Java Community and Overview Track - March 2016Java Community and Overview Track - March 2016
Java Community and Overview Track - March 2016
 
Java Community News - September 2015
Java Community News - September 2015Java Community News - September 2015
Java Community News - September 2015
 
Creating a Developer Community
Creating a Developer CommunityCreating a Developer Community
Creating a Developer Community
 
Skipping OpenStack Releases: (You Don't) Gotta Catch 'Em All
Skipping OpenStack Releases: (You Don't) Gotta Catch 'Em AllSkipping OpenStack Releases: (You Don't) Gotta Catch 'Em All
Skipping OpenStack Releases: (You Don't) Gotta Catch 'Em All
 
Ma forge++ : @Cloud
Ma forge++ : @CloudMa forge++ : @Cloud
Ma forge++ : @Cloud
 
Mastering DevOps With Oracle
Mastering DevOps With OracleMastering DevOps With Oracle
Mastering DevOps With Oracle
 
kolla
kollakolla
kolla
 
Bdd with Visual Studio 2010, Spec Flow and WatiN
Bdd with Visual Studio 2010, Spec Flow and WatiNBdd with Visual Studio 2010, Spec Flow and WatiN
Bdd with Visual Studio 2010, Spec Flow and WatiN
 
Change Management for Oracle Database with SQLcl
Change Management for Oracle Database with SQLcl Change Management for Oracle Database with SQLcl
Change Management for Oracle Database with SQLcl
 
Apache Spark Operations
Apache Spark OperationsApache Spark Operations
Apache Spark Operations
 
August Webinar - Water Cooler Talks: A Look into a Developer's Workbench
August Webinar - Water Cooler Talks: A Look into a Developer's WorkbenchAugust Webinar - Water Cooler Talks: A Look into a Developer's Workbench
August Webinar - Water Cooler Talks: A Look into a Developer's Workbench
 
Opening opensource : The Jenkins Way
Opening opensource : The Jenkins WayOpening opensource : The Jenkins Way
Opening opensource : The Jenkins Way
 
Melhore o Desenvolvimento do Time com DevOps na Nuvem
Melhore o Desenvolvimento do Time com DevOps na NuvemMelhore o Desenvolvimento do Time com DevOps na Nuvem
Melhore o Desenvolvimento do Time com DevOps na Nuvem
 
Intel Cloud Foundry and OpenStack
Intel Cloud Foundry and OpenStackIntel Cloud Foundry and OpenStack
Intel Cloud Foundry and OpenStack
 
Mastering DevOps with Oracle
Mastering DevOps with Oracle Mastering DevOps with Oracle
Mastering DevOps with Oracle
 

Recently uploaded

Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
FIDO Alliance
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
FIDO Alliance
 
Microsoft BitLocker Bypass Attack Method.pdf
Microsoft BitLocker Bypass Attack Method.pdfMicrosoft BitLocker Bypass Attack Method.pdf
Microsoft BitLocker Bypass Attack Method.pdf
Overkill Security
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
Wonjun Hwang
 

Recently uploaded (20)

The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
JohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptxJohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptx
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data Science
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform Engineering
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage Intacct
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptx
 
Microsoft BitLocker Bypass Attack Method.pdf
Microsoft BitLocker Bypass Attack Method.pdfMicrosoft BitLocker Bypass Attack Method.pdf
Microsoft BitLocker Bypass Attack Method.pdf
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptx
 
How to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in PakistanHow to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in Pakistan
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development Companies
 
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 

Scratching your own itch

  • 1. November 2011 Scratching your own itch Joey Echeverria | Solutions Architect joey@cloudera.com @fwiffo
  • 2. About Joey •  Solutions Architect •  6 months •  3+ years 2 ©2011 Cloudera, Inc. All Rights Reserved.
  • 3. Why Apache Sqoop (incubating)? •  Data center integration •  Extensible –  Sources –  Destinations •  Company specific •  Open source –  Yay! 3 ©2011 Cloudera, Inc. All Rights Reserved.
  • 4. Getting the code •  Apache SVN –  https://svn.apache.org/repos/asf/incubator/ sqoop/ •  Apache GIT –  http://git.apache.org/sqoop.git •  GitHub –  https://github.com/apache/sqoop.git 4 ©2011 Cloudera, Inc. All Rights Reserved.
  • 5. Fork you 5 ©2011 Cloudera, Inc. All Rights Reserved.
  • 6. Building the code •  Requirements –  Ant 1.7.1 –  JDK 1.6 –  Maven •  SQOOP-306 •  Targets –  jar –  test –  checkstyle 6 ©2011 Cloudera, Inc. All Rights Reserved.
  • 7. Checkstyle 7 ©2011 Cloudera, Inc. All Rights Reserved.
  • 8. IDE 8 ©2011 Cloudera, Inc. All Rights Reserved.
  • 9. Testing •  JUnit •  By hand –  ant package –  cd build/sqoop-1.5-incubating-SNAPSHOT –  bin/sqoop 9 ©2011 Cloudera, Inc. All Rights Reserved.
  • 10. Contributing back: Why? •  Next release •  Lower long-term cost •  For the community! •  It’s easier than you think! 10 ©2011 Cloudera, Inc. All Rights Reserved.
  • 11. Fears •  People won’t want my patch –  Rare –  Small projects •  Only I need this feature –  That’s what everyone says… •  Too much process! –  A little time now, a massive headache later 11 ©2011 Cloudera, Inc. All Rights Reserved.
  • 12. Sqoop vs. HDFS •  Small project –  Less contentious –  Easier to grok the code –  Easy deployment •  Immediate feedback •  Great place to start 12 ©2011 Cloudera, Inc. All Rights Reserved.
  • 13. Contributing back: How? •  File a JIRA –  https://issues.apache.org/jira/browse/SQOOP •  Post your patch on the issue •  File a review request –  https://reviews.apache.org/groups/Sqoop/ 13 ©2011 Cloudera, Inc. All Rights Reserved.
  • 14. Posting patches 14 ©2011 Cloudera, Inc. All Rights Reserved.
  • 15. The review process •  Upload your patch •  Describe your changes •  Describe your testing •  Wait for feedback •  Repeat 15 ©2011 Cloudera, Inc. All Rights Reserved.
  • 16. What else? •  Subscribe to the mailing list –  sqoop-dev@incubator.apache.org •  Look for open issues –  https://issues.apache.org/jira/browse/SQOOP •  Get hacking 16 ©2011 Cloudera, Inc. All Rights Reserved.
  • 17. Questions? 17 ©2011 Cloudera, Inc. All Rights Reserved.