The Dark Depths of iOS [CodeMash 2011]

The Dark Depths of iOS
Chris Adamson — @invalidname
CodeMash 2011

Thursday, January 13, 2011

What We'll Dig Up

• Architectural layers of iOS and their contents
• How to find stuff
• Ask me about:
• "When would I ever use this?"
• "How would I…"


iOS Architectural Layers

Cocoa Touch

Media Layer

Core Services

Core OS


Cocoa Touch

• Application infrastructure
• Touch UI Media Layer

• High-level features Services
Core
• Likely the initial focus of new iOS developers
Core OS


Media Layer

• Graphics, Audio, and Video technologies
Cocoa Touch
• 2D graphics, 3D graphics, animation, text
• Audio capture, streaming, effects, iPod Library
access
Core Services
• Video capture, editing, effects, playback
Corelibrary
OS
• Access to photo/video


Core Services

• Foundation of Apple-specific APIs
Cocoa Touch
• Initially meant for use by Cocoa (Obj-C) and
Carbon (C++)Media OS X
on Mac Layer

• Data objects (strings, URLs)
• Threading
Corein-app purchase, etc.
OS
• SQL, XML, networking,


Core OS

Cocoa Touch
• Lowest-level Apple frameworks
Media Layer
• Security, hardware-accelerated math
• System Core Services
• Low-level kernel and UNIX APIs


A Note on Languages


Languages

Cocoa Touch

Media Layer

Core Services

Core OS


Frameworks vs. Libraries


Frameworks

• Apple’s definition: “A framework is a directory that
contains a dynamic shared library and the resources
(such as header files, images, helper applications, and
so on) needed to support that library.”

• Most Apple APIs are packaged as .framework
• Most third-party APIs are packaged as .dylib or .o


Deep default dig


Deep default dig

#import <UIKit/UIKit.h>


Deep default dig
//
// UIKit.h
// UIKit
//
// Copyright 2005-2010 Apple Inc. All rights reserved.
//

#import <UIKit/UIKitDefines.h>
#import <UIKit/UIAccelerometer.h>
#import <UIKit/UIAccessibility.h>


Deep default dig
//
// UIAccelerometer.h
// UIKit
//
// Copyright 2007-2010 Apple Inc. All rights reserved.
//

#import <Foundation/Foundation.h>
#import <UIKit/UIKitDefines.h>


Deep default dig
/*!Foundation.h
! Copyright (c) 1994-2010, Apple Inc. All rights reserved.
*/

#include <CoreFoundation/CoreFoundation.h>

#import <Foundation/NSObjCRuntime.h>

#import <Foundation/NSArray.h>
#import <Foundation/NSAutoreleasePool.h>
#import <Foundation/NSBundle.h>
#import <Foundation/NSByteOrder.h>
#import <Foundation/NSCalendar.h>
#import <Foundation/NSCharacterSet.h>
#import <Foundation/NSCoder.h>


Deep default dig
/*!CoreFoundation.h
! Copyright (c) 1998-2010, Apple Inc. All rights reserved.
*/

#if !defined(__COREFOUNDATION_COREFOUNDATION__)
#define __COREFOUNDATION_COREFOUNDATION__ 1
#define __COREFOUNDATION__ 1

#if !defined(CF_EXCLUDE_CSTD_HEADERS)

#include <sys/types.h>
#include <stdarg.h>
#include <assert.h>
#include <ctype.h>
#include <errno.h>
#include <float.h>
#include <limits.h>
#include <locale.h>
#include <math.h>
#include <setjmp.h>
#include <signal.h>
#include <stddef.h>
#include <stdio.h>


Deep default dig
/*
* Functions defined in ANSI C standard.
*/
__BEGIN_DECLS
void! clearerr(FILE *);
int! fclose(FILE *);
int! feof(FILE *);
int! ferror(FILE *);
int! fflush(FILE *);
int! fgetc(FILE *);
int! fgetpos(FILE * __restrict, fpos_t *);
char! *fgets(char * __restrict, int, FILE *);
#if defined(__DARWIN_10_6_AND_LATER) && (defined
(_DARWIN_UNLIMITED_STREAMS) || defined(_DARWIN_C_SOURCE))
FILE! *fopen(const char * __restrict, const char * __restrict)
__DARWIN_EXTSN(fopen);
#else /* < 10.6 || !_DARWIN_UNLIMITED_STREAMS && !_DARWIN_C_SOURCE */
FILE! *fopen(const char * __restrict, const char * __restrict)
__DARWIN_10_6_AND_LATER_ALIAS(__DARWIN_ALIAS(fopen));
#endif /* >= 10.6 &&_(DARWIN_UNLIMITED_STREAMS || _DARWIN_C_SOURCE) */


Your App

Cocoa Touch

Media Layer

Core Services

Core OS


Your App

<UIKit.h> Cocoa Touch

Media Layer
<Foundation.h>
<CoreFoundation.h> Core Services

<stdio.h> Core OS


#include, #import, @class
• #include literally includes all of a header file
• #import, defined by Obj-C, includes a given file only
once, intended to help with circular dependencies

• @class just promises that a given Obj-C class will be
#import’ed at some point

• Useful for circular header dependencies; one can
use @class in .h, then #import in .m


Cocoa Touch

Media Layer

Core Services

Core OS


Core Graphics (Quartz)

• C-based API for 2D graphics
• Structs and functions start with CG…
• Some UIKit methods work with struct CGRect
• Most drawing functions (stroke, fill, set path, etc.)
start with with CGContext… and take a
CGContextRef as first parameter


Core Animation
• Obj-C API for hardware-accelerated compositing,
rendering, and animation

• Don’t think of it as “just” animation
• CALayer is a presentable surface; every UIView is
backed by a layer (see UIView.layer property),
which it uses for rendering

• You could write a book on this stuff


Core Text

• C-based API for advanced text layout and font
handling

• Most apps can just use Cocoa UITextView and
UIFont

• Can render text into CGContext


Image I/O

• C-based API split out of Core Graphics
• Allows high-performance reading and writing of
many image formats

• JPEG, JPEG2000, RAW, TIFF, BMP, PNG, etc.
• Provides access to image metadata and color
management


Assets Library
• Obj-C API to access photos and videos in users library
(c.f., “Photos” app)

• Introduced in iOS 4.0
• ALAssetsGroup provides ALAssets, which provide
ALAssetRepresentations, which provide
metadata, CGImageRefs, etc.

• Save new images to photo library via
ALAssetLibrary


OpenGL ES
• C-based industry-standard API for 2D and 3D
rendering

• Not an Apple API, so code conventions are different
• All drawing is done to an EAGLContext object,
which you set up with a EAGLDrawable (the only
implementation of which is CAEAGLLayer).

• You could write a book on this too


Media Player
• Obj-C API to access user’s iPod Library
• Audio only: music, audio podcasts, audiobooks
• Use MPMediaQuery to get MPMediaItems, then play
with MPMediaPlayer, or inspect metadata
properties

• MPMoviePlayerController offers a simple movie
player; AV Foundation is better.


AV Foundation

• Obj-C API for audio and video capture, editing, and
playback, introduced in iOS 4.0

• iMovie for iPhone is apparently written with this
• Huge framework, comparable to QuickTime
• Can open audio URLs from Media Library


Core Media / Core Video
• Core Media: C API to describe byte buffers, formats,
and time for AV Foundation

• CMTime struct includes units and timescale, so it is
always exact for a media-appropriate scale

• Core Video: C API to provide pixel buffers and image
buffers for AV Foundation

• Neither is directly applicable to app developers
outside of AV Foundation


OpenAL

• C-based API for 3D spatialized sound
• Third-party API, designed to resemble OpenGL
• Create buffers of single-channel PCM, connect these
to sources, set properties on the sources (location,
orientation, etc.), then configure properties of a
listener


Core Audio
• C API for low-latency audio processing (capture and
play-out to speaker or headphones)

• Provided utility classes are in C++
• Consists of two “engines” - Audio Units and Audio
Queues - along with convenience APIs for audio file I/
O, network I/O, format conversion, etc.

• OpenAL is implemented atop CA as an Audio Unit
• Legendarily hard to use, needs a book…

Core MIDI

• Obj-C and C APIs for communicating with external
MIDI devices (musical instruments) over network or
via dock connector

• Introduced in iOS 4.2
• Does not provide a software synthesizer


Foundation
• Obj-C API with essential collection of data types
(100+ classes, 20+ protocols) used throughout Cocoa

• Collections: NSArray, NSSet, NSDictionary
• Data objects: NSString, NSDate, NSCalendar
• Primitive wrappers: NSNumber, NSValue, NSData
• Many of these are immutable, have mutable
subclasses


Foundation: Notifications
• NSNotification represents broadcastable data –
typically a “change” – as a name, source object, and
optional info dictionary

• Each app has an NSNotificationCenter to broadcast
notifications

• Mac OS X has an NSDistributedNotificationCenter
• Interested parties add themselves as observers of the
NSNotificationCenter for specific notification names
(and, optionally, sources)


Foundation: URL Loading System

• Create an NSURL
• From this, create an NSURLRequest
• From this, create NSURLConnection, providing
delegate to receive NSURLResponse

• Classes also provided for caching, authentication
• Only works with file:, http:, https:, and ftp:
URLs.


Foundation: Handy Stuff
• XML parsing: NSXMLParser (event-driven)
• Bonjour: NSNetService, NSNetServiceBrowser
• NSScanner: substring matching and extraction
• NSUndoManager
• NSKeyedArchiver, NSKeyedUnarchiver
• NSSortDescriptor: Used by sort methods in
NSArray and NSMutableArray


Foundation’s C Stuff
• NSAssert…()
• NSLocalizedString…(): gets localized string from
app bundle

• NSLog()
• NSRange: Struct used for substrings in NSStrings.
Comes with helper functions like NSMakeRange(),
NSEqualRanges(), NSIntersectionRange()…


Foundation: NSObject

• Polymorphism: isKindOfClass:,
respondsToSelector:, conformsToProtocol:

• Reference-counting: retain, release,
autorelease

• Fundamental rule: you own any object you create
with alloc, new, or copy, and must eventually
release (or autorelease) it.


Core Foundation
• C API originally intended to provide common services
to Cocoa (Obj-C) and Carbon (C++) on Mac OS X

• Defines class-like “opaque types”; instances of these
are still called “objects”.

typedef const struct __CFString * CFStringRef;

• CF objects use same reference-counting scheme as
Foundation: CFRetain(), CFRelease(), but no
autorelease.


Toll Free Bridging

• Many CF opaque types are effectively identical to
Foundation classes, and can be cast with zero cost

NSString *myString = @"My Foundation string";
CFStringRef myCFString = CFSTR ("My Core Foundation string");
CFStringRef hisString = (CFStringRef) myString;
NSString *hisCFString = (NSString*) myCFString;


Toll-Free Bridged Classes
From cocoadev.com:
NSArray = CFArray
NSMutableArray = CFMutableArray
NSCalendar = CFCalendar
NSCharacterSet = CFCharacterSet
NSMutableCharacterSet = CFMutableCharacterSet
NSData = CFData
NSMutableData = CFMutableData
NSDate = CFDate
NSDictionary = CFDictionary
NSMutableDictionary = CFMutableDictionary
NSNumber = CFNumber
NSTimer = CFRunLoopTimer
NSSet = CFSet
NSMutableSet = CFMutableSet
NSString = CFString
NSMutableString = CFMutableString
NSUrL = CFURL
NSTimeZone = CFTimeZone
NSInputStream = CFReadStream
NSOutputStream = CFWriteStream
NSAttributedString = CFAttributedString
NSMutableAttributedString = CFMutableAttributedString


Not Toll-Free Bridged
From cocoadev.com:

NSBundle != CFBundle
NSHost != CFHost
NSRunLoop != CFRunLoop
NSNotificationCenter != CFNotificationCenter
NSSocket != CFSocket

Note: NSHost does not exist on iOS, and NSSocket doesn’t exist (or
isn’t public) on iOS or OS X.


Core Foundation Conventions
• Core Foundation functions typically name the opaque type
they take as a first (or second) argument.

• Notice similarity between Foundation (Obj-C) and Core
Foundation calls:
! int i = [myString length];
! int j = CFStringGetLength(myCFString);

! NSArray *components = [myString componentsSeparatedByString:@" "];
! CFArrayRef cfComponents =
! ! CFStringCreateArrayBySeparatingStrings (kCFAllocatorDefault,
! ! ! ! ! ! ! ! ! ! ! ! ! ! myCFString,
! ! ! ! ! ! ! ! ! ! ! ! ! ! CFSTR(" "));

• Important difference here: you own and must
CFRelease() the cfComponents variable.


CF Exclusives
• Some Core Foundation APIs have no equivalent classes
or similar functionality in Foundation

• CFBagRef: unordered set that allows duplicates
• CFStringTokenizerRef (compare to NSScanner)
• CFTreeRef: tree-structured data
• CFUUIDRef: Universally Unique Identifier (RFC
4122)


CFNetwork
• Much deeper networking API than Foundation’s URL
Loading System

• Provides CF-friendly abstraction over BSD sockets
(client or server)

• APIs for DNS host name lookup, read and write
streams

• Also has a CF version of Bonjour


Threads
• NSThread provides Obj-C semantics for executing code
in a separate thread of execution

• Subclass NSThread and override main, or use
initWithTarget:selector:object:

• All UIKit code should be running on the “main” thread.
• See +[NSThread mainThread], +[NSThread
isMainThread], +[NSObject
performSelectorOnMainThread:withObject:w
aitUntilDone:]


Run Loops
• An event processing loop attached to a single thread
• Inputs to the loop come from NSPorts (distributed
messaging), NSTimers, custom sources, and
performSelector:onThread:… calls.

• Main thread’s run loop is set up by UIApplication.
You need to create your own for custom threads.

• If a method asks for a run loop “mode”, you almost
always want NSDefaultRunLoopMode (aka
kCFRunLoopDefaultMode)


NSOperation

• Abstract class that defines a single task that may be
run concurrently, or as a consequence of one or more
other operations completing

• Generally used with an NSOperationQueue, which
will determine what thread to call your operation on


Blocks
• Closures for C — code plus stack and heap variables.
• Allows you to pass state and the code to work with it
• New Apple frameworks prefer blocks to delegates and
other asynchronous design patterns (e.g., take a
completionHandler block argument, to be executed
when the asynchronous task completes)

• Can/should create an NSOperation with a block
• Hope you saw Daniel Steinberg’s session earlier

Grand Central Dispatch
• Multicore-savvy programming for Mac and iOS, in C
• Based on a “queue of blocks” metaphor
• Can dispatch your blocks on main queue (main thread),
concurrently, or serially

• Can group blocks and get notification when they complete,
even if they run on different threads

• GCD is defined in Core Services, but typically used directly
only by lower-level programmers (consider NSOperation
instead)


System Configuration
• Another C API. Only iOS functionality:
SCNetworkReachability

• Synchronous check:
SCNetworkReachabilityGetFlags()

• Asynch: SCNetworkReachabilitySetCallback()
• Apple always tests what your app does when the
network goes away (e.g., Airplane Mode). You will be
rejected if your app bricks without network.


Other Apple Core Services
• Store Kit: Obj-C API for In-App Purchasing
• Core Location: Obj-C API for determining current
location from device features (GPS, cellular radio),
reckoning course, distances, etc.

• Quick Look: Obj-C API allows apps to provide
thumbnails of documents for previewing

• Address Book: C API to access names, addresses, …
• Core Data: Obj-C API for object-relational mapping

3rd Party Core Services

• SQLite3: C API for simple file-based relational
database

• Libxml2: C API for XML parsing, DOM-style or SAX-
style

• Documentation for these are on their respective
websites, not in Xcode


Depth Check

You Are
Here


Accelerate
• Hardware-accelerated math, big number, and DSP
functions, in C

• LAPACK (Linear Algebra Package) – high-level LA,
solve linear systems, Eigenvalues and Eigenvectors

• BLAS (Basic Linear Algebra Subroutines) – low-level
LA, matrix, matrix-vector, matrix-matrix

• vDSP – basic math on arrays, convolution and
correlation, Fast Fourier Transform

• float faster than double, esp. on ARMv7

External Accessory
• Requires paid membership in “Made for iPod /
Works with iPhone” program

• If you thought App Store was picky…
• Three Obj-C classes and one delegate protocol:
Iterate over available wired and Bluetooth
EAAccessory objects with EAAccessoryManager,
create an EASession and access its inputStream
and outputStream.


Keychain Services
• API to store user passwords, purchase history, other
sensitive data.

• C functions to query, add, update, delete CFDictionary
objects containing key-value pairs

• Each dictionary is one user item. Pairs are things like an
account name, the item type, the item’s ID, the item’s
value, etc.

• Keychain survives app deletion and re-install.
• “Friendly” apps can share Keychain data

Other Security Services
• C functions for managing certificates, public and private
keys, representing trust policies

• Match certificates to private keys
• Create and request certificate objects
• Import certificates, keys, and identies
• Create public-private key pairs
• Randomization Services provides crypto-secure random
numbers


System

• Low-level UNIX interfaces
• No third-party access to kernel or drivers
• No Xcode documentation (other than code
completion)

• Investigate man pages, headers in <iOS_SDK>/usr/
include, or web-based documentation


man page
Yuna:~ cadamson$ man memset
MEMSET(3) BSD Library Functions Manual MEMSET(3)

NAME
memset -- fill a byte string with a byte value

LIBRARY
Standard C Library (libc, -lc)

SYNOPSIS
#include <string.h>

void *
memset(void *b, int c, size_t len);

DESCRIPTION
The memset() function writes len bytes of value c (converted to an
unsigned char) to the byte string b.

RETURN VALUES
The memset() function returns its first argument.

SEE ALSO
bzero(3), memset_pattern(3), swab(3)
:


man page
Yuna:~ cadamson$ man memset


System - C Standard Library
• stdio.h – Standard I/O: open, close, get data from / put
data to FILEs (not necessarily a flat file)

• stdlib.h – General-purpose functions: malloc()/
free(), type conversions, qsort(), etc.

• string.h & wchar.h — C string utilities
• math.h – Basic math constants and functions
• And more… http://en.wikipedia.org/wiki/
C_standard_library or just read K&R.


System - BSD Sockets

• sys/socket.h, netinet/in.h, netinet6/in6.h, etc.
• Create socket with socket()
• Client-side: connect(), then send()/recv() or
read()/write()

• Server-side: bind(), accept(), then send()/recv
() or read()/write()


System - POSIX Threads
• pthread.h – functions for creating and using threads,
thread-safety via mutexes, conditions, and
synchronization

• Create thread with pthread_create(), passing a
pthread_t* to receive the created thread, thread
attributes (NULL for default), function pointer to run,
and void* to pass as argument to the function

• Can use sys/semaphore.h as an alternate thread-
safety mechanism


Other neat stuff in /usr/include
• 150 files and directories, including
• zlib.h, tar.h – Data compression
• CommonCrypto/ – MD5, SHA, other crypto
• regex.h – Regular expressions (Standard C library)
• asl/ – Apple System Log access
• However, not everything in /usr/lib has a public
header (e.g., libbz2.dylib, libtidy.dylib)


Where to?

Cocoa Touch

Media Layer

Core Services

Core OS


Portability considerations
• POSIX code is easy to port to/from iOS
• Con: C strings are ASCII, stdio & inet calls block
• Core Foundation doesn’t port
• But data types are much richer, network and I/O
APIs are asynchronous by design

• How far does POSIX get you on mobile anyways
(Android, PalmOS, Windows Phone 7)?


Developer considerations

• Most iOS developers did not start as C programmers
• Came from Java, JavaScript, Windows, etc.
• Can be tricky to context-switch between Obj-C and C
• For switchers, Foundation is easier than Core
Foundation, and Core Foundation is easier than
standard C libraries


What’s Next

Cocoa Touch

Media Layer

Core Services

Core OS


Future directions
• Much of Mac OS X has now been ported to iOS.
Interesting stuff that hasn’t been:

• Speech synthesis/recognition (c.f., App Kit)
• Already available to Apple’s apps
• Ink services (handwriting recognition)
• WebServicesCore (XML-RPC, SOAP)
• Non-hacky resolution independence

Q&A

• invalidname [at] gmail [dot] com
• @invalidname
• http://www.subfurther.com/blog


The Dark Depths of iOS [CodeMash 2011]

Recommended

Recommended

More Related Content

Similar to The Dark Depths of iOS [CodeMash 2011]

Similar to The Dark Depths of iOS [CodeMash 2011] (20)

More from Chris Adamson

More from Chris Adamson (20)

Recently uploaded

Recently uploaded (20)

The Dark Depths of iOS [CodeMash 2011]