In this talk, I will review a new application in the volunteer computing project LHC@home, called Test4Theory, which is currently being alpha tested at CERN. This application uses virtual machine technology and a range of other software tools to handle large, complex software environments that are rapidly evolving, so that they can be distributed using BOINC.
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
LHC@home and CernVM – a new approach to porting large-scale applications to BOINC
1. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
LHC@home and CernVM – a new approach to
porting large-scale applications to BOINC
˜ ´
Daniel Lombrana Gonzalez, Ben Segal
and Artem Harutyunyan
Citizen Cyberscience Centre
CERN
March 20, 2011
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
3. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
The beginning
The challenge
Why don’t you run real LHC physics on BOINC?
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
4. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
The challenge
Allow “any” PC to run a full LHC physics application.
Make those commodity computers look like a “standard”
CERN Data Center.
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
5. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
We did it!
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
7. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
Challenge 1: Porting source code
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
8. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
Challenge 1: Porting source code
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
9. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
Challenge 1: Porting source code
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
10. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
Challenge 2: Job management systems
Jobs must be fed into BOINC PCs, but CERN physics
experiments have their own and don’t want to use the
BOINC distributions system.
Volunteer computing resources are not “managed” and
thus cannot be “trusted”.
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
12. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
CernVM + Co-Pilot
Using Virtualization and CernVM it
is possible to solve the challenges
(1) for application deployment on
volunteers’ machines.
With CernVM’s Co-Pilot it is possible
to connect existing Grid
infrastructures of LHC experiments
with the BOINC volunteer resources,
solving challenges (2).
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
13. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
CernVM solution
CernVM is
a baseline Virtual Software Appliance for the participants of
CERN LHC experiments.
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
14. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
CernVM solution
CernVM is
a baseline Virtual Software Appliance for the participants of
CERN LHC experiments.
The goal is
to remove a need for the installation of the experiment software
and to minimize the number of platforms.
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
15. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
CernVM Co-Pilot solution
A framework for the delivery and execution of the workload
on the remote virtual machines.
Consists of components developed to ease the integration
of cloud resources into existing Grid infrastructures.
Components communicate using Jabber/XMPP instant
messaging protocol.
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
17. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
CernVM
Using virtualization and CernVM solve the problem of
porting the source code and deploying over PCs.
CernVM Co-Pilot can connect existing Grid infrastructure
of LHC experiments with BOINC resources.
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
18. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
BOINC + CernVM infrastructure
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
19. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
Co-Pilot Adapters
Each LHC experiment that wants to connect to a set of
CernVM machines via Co-Pilot, needs an adapter.
Each adapter fetches jobs from the experiment’s preferred
job scheduler, submits them to the CernVM machines, and
returns the results to the scheduler.
Co-Pilot has a built-in security barrier, preventing untrusted
CernVM machines to access Grid resources.
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
21. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
Monte Carlo events generation
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
22. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
Alpha testing
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
23. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
Active forums
More than 500 posts in the
forums so far.
Very active volunteers,
reporting all the problems they
find.
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
24. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
High schools
Three young students were invited to test the project.
One of them became very helpful in the first stages of the
testing phase.
The obtained feedback allowed us to improve the project,
knowing that young students could collaborate with
complex projects.
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
25. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
Numbers
Since the beginning of the project 25000 jobs have been
successfully executed.
80% of the jobs were set to generate 300000 events (the
rest were smaller ones that generated 50000 events).
Totaling 6250 millions of events simulated since December
2010.
There are on average 20 on-line users at any given point in
time (peaks up to 30 users).
We are running 1200-1300 jobs daily.
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
27. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
Conclusions
A custom virtualized execution environment has been
added to BOINC.
Very complex and unmodified applications can be run
within the system.
Adapting Grid services to Co-Pilot is now understood.
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
28. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
Conclusions
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
29. Introduction
Technical challenges
Employed solutions
CernVM + Co-Pilot
Test4theory project
Conclusions
Questions
teleyinex@gmail.com
Icons from Tango and Gnome Desktop projects (Creative Commons & GPL License)
˜
D. Lombrana, B. Segal & A. Harutyunyan LHC@Home