Async-await best practices in 10 minutes

Editor's Notes

1. * Let me put that more strongly * For goodness' sake, stop using async void everywhere. * (At first that was going to be the title of my talk)
2. * We've seen that async void is a "fire-and-forget" mechanism * Meaning: the caller is *unable* to know when an async void has finished. * And the caller is *unable* to catch exceptions from an async void method. * Guidance is to use async void solely for top-level event handlers. * Everywhere else in code, like SendData, async methods should return Task. * There’s one other danger, about async void and lambdas, I’ll come to it in a moment.
3. * Basically the problem boils down to events. * Building a complicated UI that responds to events. * The developer had tried one approach, keeping the event handlers local. * That led to a nightmare of nested lambdas. * We tried another approach, figuring out the game's state machine, and how it responds to events. * That ended up being too global. That is, things that should have been local ended up being part of the global state machine. What should have been local variables were promoted to class fields. * The thing is, we have to figure out a way to tame events. * Events have been with us for a long time, and they'll be with us for a long time to come. * WPF, Silverlight, even in Windows8, they're still full of events. * The challenge is how to tame them. * I'm going to show how we can tame them by wrapping them up with tasks.
4. * So let's review. * It's vital to distinguish between what is CPU-bound work and what is IO-bound work. * CPU-bound means things like LINQ-to-objects, or iterations, or computationally-intensive inner loops. * Parallel.ForEach and Task.Run are good ways to put these CPU-bound workloads on the threadpool. * But it's important to understand that threads haven't increased scalability * Scalability is about not wasting resources * One of those resources is threads * Let's say your server can handle 1000 threads. * If you had 1 thread per request, then you could handle 1000 requests at a time. * But if you created 2 threads per request, then only 500 requests at a time.
5. * Oh. Just hold on there a moment. * What the heck kind of deserialization takes so long? 100ms per house? That's an eternity. * Well, I checked with the developer. * Turns out his deserialization wasn't really what I'd call deserialization. * It was looking up tables in a database. * That's why it took so long. It was network-bound, not CPU-bound.
6. * So let's review. * It's vital to distinguish between what is CPU-bound work and what is IO-bound work. * CPU-bound means things like LINQ-to-objects, or iterations, or computationally-intensive inner loops. * Parallel.ForEach and Task.Run are good ways to put these CPU-bound workloads on the threadpool. * But it's important to understand that threads haven't increased scalability * Scalability is about not wasting resources * One of those resources is threads * Let's say your server can handle 1000 threads. * If you had 1 thread per request, then you could handle 1000 requests at a time. * But if you created 2 threads per request, then only 500 requests at a time.
7. * We've seen that async void is a "fire-and-forget" mechanism * Meaning: the caller is *unable* to know when an async void has finished. * And the caller is *unable* to catch exceptions from an async void method. * Guidance is to use async void solely for top-level event handlers. * Everywhere else in code, like SendData, async methods should return Task. * There’s one other danger, about async void and lambdas, I’ll come to it in a moment.
8. * There are two ways of thinking about asynchrony. * First is from the method signature, the contract, how people will call it:* Synchronous signature means people expect to call it, to perform an operation here and now on this thread, and they won't get back control until it's done. * Asynchronous signature means people expect to call it to kick off an operation, but they'll get back control immediately to do whatever else they want. * Second way to look is from how it's actually implemented* An implementation that's synchronous is one that burns up a CPU core, doing work* An asynchronous implementation is one that's lighter, it barely touches the CPU, it just does small bits of work and schedules more. * The thing is, your callers will look at your signature of your method, and they'll make assumptions right or wrong about how you're implemented underneath. * It'll be your job to stay in line with those expectations.
9. * So what are the assumptions people will make? * Imagine someone comes up to your API. * They're going to read the documentation. * Hah! Who am I kidding? They might read the XML doc-comments if we're lucky. * They're going to say "Hey, here's a method, its name ends with Async, so..."* Maybe I'm in a server app. I bet this method's not going to spawn new threads. * I can trust this method to be a good citizen on my server. [CLICK] * I also know that I can parallelize it. * Maybe it's a download API. I can kick off 10 downloads simultaneously, just by invoking it 10 times and then awaiting Task.WhenAll. * And it's not going to be hurting my scalability to do so. [CLICK] * These are the assumptions people make when they see "Async" at the end of your library method. * What you have to ask yourselves is this: is it actually true for the async methods you're creating?
10. * We're going to look at an example where this isn't actually true.   [VS] TreadPoolScaling.Run() // uncomment Go To Definition * There are going to be a number of small demos in this talk. * This first one is about what happens when a library uses Task.Run internally. * Simple app, console app, but I'm just spinning up a Winforms dialog here. * This is the minimal code I need to get a UI message-loop. (don't want rest of plumbing)   class IO * It's a demo of a library, so we'll have three layers: the app that uses the library then the library itself then the underlying framework functionality that the library uses. * In this case, just as simulation, my OS provides two forms of its DownloadFile API * One of them's truly synchronous - it really does tie up the thread. Doesn't burn the CPU, but does block the CPU. * The other one's truly asynchronous - not using CPU, not tying up Thread.   class Library * Here's one way to write the library. It wants to offer up an asynchronous API * And in this version, it's using the synchronous OS API. Maybe that's the only one available. * So to become async, my API needs to wrap it, with await Task.Run * It's the top-right quadrant. It looks async, but it's wrapping an implementation that's synchronous. * Probably to avoid blocking the callign thread.   b.Click += async delegate * And here's what the app developer wrote, the user of my library. * They want to be asynchronous, they want to stay responsive. * But say they don't want just one, but they want to download 100 files. * They saw that it was an async method, so they trusted they could just kick off all the tasks and then await Task.WhenAny   [RUN] * Now it has kicked off all 100 of those tasks. * But because each one wants to use a background thread, it's actually going in bursts. * I have four logical cores on this laptop, so the threadpool starts by giving me four threads. As many threads as we have cores. * Then it looks a second later, says it looks like you've made crappy use of those threads, most of them were idle, waiting on IO * So it looks like you need more threads   [RUN] * See the first batch was 4, then next batch was 5, then 6 * The threadpool has this predefined scaling behavior, hill-climbing * So I've had to wait until the threadpool catches up to me, until it eventually finds its optimal number. * But actually my app didn't need any threads. * As an app author, I didn't even think any threads were involved. * That's the key. You don't want to go messing with things that aren't yours, global resources. * And the threadpool is one of those things. * It belongs to the app developer, not to you the library author. * They might have their own ideas about how they want to use the threadpool.   var contents = await IO.DownloadFileAsync() * Now this one's pure async   [RUN] * And this time all 100 files can download at the same time. * This is what we'd expect. * I shouldn't have to block waiting for the threadpool to grow * I just have the assumption that I'm just kicking off work from the UI thread. * You don't want to be a library author who violates that assumption [CLICK] * If your library's using Task.Run, you're putting in roadblocks that prevent the app from using its threads effectively
11. * So when you do await, it first captures the current SyncContext before awaiting. * When it resumes, it uses SyncContext.Post() to resume "in the same place" as it was before [CLICK] * For app code, this is the behavior that you almost always want. * When you await a download, say, you want to come back and update the UI. [CLICK] * But when you're a library author, it's rarely the behavior that you want. * You usually don't care which threading context you come back on, inside your library methods. * It doesn't matter if your library method finishes off on a different thread either, maybe the IO completion port thread. * That's because when the user awaited on your library method, then their own await is going to put them back where they wanted. They don't need to rely on you.
12. * Sometimes you think, "well I've implemented FooAsync(). shall I also implement Foo()? Not everybody's going to want to await." * Why don't I just give them a helper, with the same implementation as FooAsync, but with a contract that's synchronous? [CLICK] * User's going to see that there are two versions of your API, one synchronous, one asynchronous. * They'll assume the sync version must be faster than the async, otherwise why else would it be defined? [CLICK] * Also, suppose I'm the UI thread or complicated code where for whatever reason I don't want to await. Or suppose I'm on a constructor or one of those other places where I can't await. Maybe it'll be quick enough just to take 5-10ms to call the sync version from the UI thread, because my domain-specific knowledge knows it'll be okay. [CLICK] * That's what user will think when see that you have both sync and async. * I wrote this sync version at the bottom. It's the wrong thing to do. It violates that assumption. * Worse than that, it will actually DEADLOCK! * That's because the call to Wait blocks the UI thread, and prevents any awaits from resuming. * So, don't do it! Don't ever use .Wait() or .Result in a library method if you might be called on the UI thread.
13. * Principle is that the threadpool is a global resource. * You as a library developer have to play nice, help the app author use their domain-specific knowledge as to how to create threads. * Don't do it yourself. * Only show an async signature when you r method is truly async. * Don't block in your library calls. * If you block on the UI thread, disaster. * If you block on a threadpool thread, you're hurting the threadpool.
14. * demo...   [VS] CapturingContext * There's a structure to how I do these perf demos   const int ITERS = 20000; * Repeat inner loop 20,000 times   await t * This one does the defaut - it does capture and resume on the captured synchronization context   await t.ConfigureAwait(false) * This one, same code, just resumes on whichever thread it left off. Likely the threadpool thread.   await Task.WhenAll(WithSyncCtx(), WithoutSyncCtx()); // warm-up * Run each method once to ensure it's JIT'd   [RUN, CTRL+F5] * We see a difference ten-fold difference. * If doing the loop 20,000 times, it adds up to half a second. * That's not much, just a few microseconds. * And it's completely irrelevant if you're only doing 10 or 100 awaits in your library method * But if you have an await inside your inner loop, or if your user will call you inside their inner loop, that's when it adds up [CLICK] * So your habit as library-authors should be: * as a habit, always use ConfigureAwait(false) * There are almost no cases where you have to jump back to the thread where you were
15. * I need to get technical. Talk about "SynchronizationContext". * It represents a target for work * Has a core method called "Post" * You invoke Post to get back into a particular context. [CLICK] * For example, in Winforms, if you get the current SynchronizationContext and do Post on it, it does a Control.BeginInvoke. That's how Winforms gets onto the UI thread. [CLICK] * And in WPF/Silverlight/Win8 it's similar, the DispatcherSynchronizatonContext. When you do a Post on it, it uses Dispatcher.BeginInvoke. [CLICK] * And ASP.Net current synchronization context, when you do Post() on it, it schedules work to be done in its own way. * There are about 10 in the framework, and you can create more. * And this is the core way that the await keyword knows how to put you back where you were.
16. * So when you do await, it first captures the current SyncContext before awaiting. * When it resumes, it uses SyncContext.Post() to resume "in the same place" as it was before [CLICK] * For app code, this is the behavior that you almost always want. * When you await a download, say, you want to come back and update the UI. [CLICK] * But when you're a library author, it's rarely the behavior that you want. * You usually don't care which threading context you come back on, inside your library methods. * It doesn't matter if your library method finishes off on a different thread either, maybe the IO completion port thread. * That's because when the user awaited on your library method, then their own await is going to put them back where they wanted. They don't need to rely on you.
17. * And so in the framework we provide this helper method Task.ConfigureAwait. * Use it on your await operator. * Default, true, means the await should use SynchronizationContext.Post to resume back where it left off * If you pass in false, then if possible it'll skip that and just continue where it is, maybe the IO completion-port thread * Let's just stay there! is as good a place as any! [CLICK] * If your library doesn't do this, and you're using await in an inner loop, then you're waisting the user's message-loop * being a bad citizen, flooding THEIR UI thread with messages that don't have anything to do with them
18. * So the principle is, you've got to be aware that your library can be called from different environments * SynchronizationContext: think how will your library method run if it's called from the UI context? or from the threadpool context? or ASP.Net? * Guidance: use ConfigureAwait(false): it's good for perf, and also can avoid deadlocks
19. * We've talked about perf considerations. * Async is as fast as it can be, and the inherent overheads are only noticeable in a tight inner loop. We're talking millions of iterations, not just a few hundred or thousand. * If we can't help, and our API has to be called frequently, there are some great built-in perf features * First, there was the "Fast Path". If an await has already completed, then it just plows right through it. * And if you get to the end of the method without any "slow-path" awaits, then you avoid a bunch of memory allocations. * Guidance is, try to avoid chatty APIs. Make APIs where the consumer of your library doesn't have to await in an inner loop. * You can GetNextKilobyteAsync() instead of GetNextBitAsync(). * If you have to, we saw how to cache the returned Task<T> to remove the one last allocation on the fast path. * Using your domain knowledge that YOU have about the nature of YOUR API can let YOU decide to cache tasks in a way that makes sense.
20. * Those perf optimizations around await are just there, you'll benefit from them automatically if you use the fast path. * But the one I talked about earlier, about caching the returned Task if it's not one of the common ones, that requires some work on your part.   [VS] ReadAsync() * Here I'm going to show you a typical pattern you can use to cache the returned Task, to avoid having to allocate a new Task object every single time.   byte[] data = new byte[0x10000000[ * I'm going to allocate a quarter of a gig, and measure how many allocations it needs to copy it.   input.CopyToAsync(Stream.Null).Wait(); * For the copying, I'll be using the .NET framework method CopyAsync   int newGen0 = GC.CollectionCount(0) * And I'll be measuring how many times the GC had to run   class MemStream1 : MemoryStream * What I'm testing is two different implementations of MemoryStream   return Read(buffer, offset, count); * My test used Stream.CopyAsync, so I know its going to call into ReadAsync * This first implementation is just a simple async method * no awaits, so it always takes the fast path * But it returns the number of bytes read. * This isn't one of the common values, so it's not a singleton. * Instead it's going to allocate a new Task object every time this is called. * It happens that Stream.CopyAsync is using buffers of size 80k each time, so every Task<int> that it allocates will be Task<int> with value 81920 * But it's still allocating a new copy of that every single time.   private Task<int> m_cachedTask; * Let's look at this second memory-stream implementation. * This one keeps a cache of the last Task<int> it returned. * When Stream.CopyAsync invokes ReadAsync, it knows what value integer it needs to return as a Task.   if (m_cachedTask != null && m_cachedTask.Result == numRead) * And if the Task it's cached has the right value, well, it might as well return that. * A single Task object can be used as many times as you like after it's been completed. * After the Task has completed, it's immutable.   m_cachedTask = Task.FromResult(numRead); * But if the cache wasn't there, or had the wrong value, * then we'll generate an already-completed task with the right value. * That's what Task.FromResult does.   [RUN, CTRL+F5] * And there we see that we've saved an appreciable number of allocations. * I want to stress, it doesn't cache the last INTEGER it returned. * That'd miss the point. Our goal is to reduce the number of Task objects we allocate. * So we have to cache the Task<int>, not just the int. * One thing to ask, how big should our cache be? * Here I've just used a single-element cache. It only stores the previous one. * And what you'll find is that, generally, just a single-element cache works great!   TrackGcs(new MemoryStream(data)); * I just wanted to show you some more perf numbers. * Here I'm going to use the standard built-in MemoryStream   [RUN IT, CTRL+F5] * What we see is that MemoryStream actually has some further internal optimizations to eliminate all GCs in this test. * You can go a long way. It's a question of how much time you want to spend as a library author, and how frequently your library APIs will be used, for how worthwhile it is.   [CLICK] * Using your domain knowledge that YOU have about the nature of YOUR API can let YOU decide to cache tasks in a way that makes sense. * We used it in the .NET framework to dramatically improve the performance of BufferedStream and MemoryStream. [CLICK] * And we saw that it's important to cache the Task<T>, not the T * And a cache size of just "1" is often the right choice!
21. * We've talked about perf considerations. * Async is as fast as it can be, and the inherent overheads are only noticeable in a tight inner loop. We're talking millions of iterations, not just a few hundred or thousand. * If we can't help, and our API has to be called frequently, there are some great built-in perf features * First, there was the "Fast Path". If an await has already completed, then it just plows right through it. * And if you get to the end of the method without any "slow-path" awaits, then you avoid a bunch of memory allocations. * Guidance is, try to avoid chatty APIs. Make APIs where the consumer of your library doesn't have to await in an inner loop. * You can GetNextKilobyteAsync() instead of GetNextBitAsync(). * If you have to, we saw how to cache the returned Task<T> to remove the one last allocation on the fast path. * Using your domain knowledge that YOU have about the nature of YOUR API can let YOU decide to cache tasks in a way that makes sense.
22. Telerik Luis abreu Pluralsight Nokia redgate
23. Para quem puder ir preenchendo, assim não chateio mais logo  É importante para recebermos nós feedback, e para darmos feedback aos nossos oradores http://goqr.me/