The 2019 approach

The approach in 2019 relied on a pipeline of realtime text detection, recognition, and machine translation to produce a translated version of text from the game as it appeared. That translated text would then be displayed in a separate window. Even with the use of an Azure Custom Translate model (trained off the fan translation of an earlier game in the series), machine translation quality was variable, and the ergonomics of the whole setup were simply too awkward to apply for the entirety of a 30,000+ line visual novel.

The 2025 approach

In the session, we looked at how the use of Claude Code significantly improved the approach and outcomes in a second, "vibe translated" attempt. Agentic assistance greatly simplified the process of reverse engineering the game script and generating extraction/reinsertion and repacking utilities, enabling a translation to be applied directly to the game. The switch from machine translation of isolated lines to an llm-based, 'context-aware' translation of batches in sequence, including a multi-pass review process and automatically-maintained translation consistency guide, resulted in a (subjectively) greatly improved quality of translation. The development of a parallel agent framework enabled reliable, unattended, and high throughput translation of the script, in a manner that took advantage of the Claude Code Max subscription usage windows to perform an estimated $1,100 AUD of token usage at zero incremental cost.

Although much code was produced and executed in the process of reverse-engineering, extraction, translation and repacking of the game - no code was written by me. I noticed various suboptimal implementations as I reviewed some of the scripts, prompts, and intermediate outputs while pulling together the slides; but when vibing, "we run the code - we don't judge". The hard work was handled by my good friend Claude Code.

Despite the fact that at first glance this appears to be an excellent and compelling approach towards making a Japanese game accessible to an English speaker, I have to acknowledge that I have no idea of how good or bad the translation output actually is, since I can't read Japanese to verify it. This ties in to a thought I have around cautioning the use of AI to do something you aren't able to do yourself. Beyond the risk involved in taking a dependency on a volatile third-party, it's worth considering that you are unlikely to be able to properly evaluate the quality, completeness, or robustness of an artefact related to a domain you do not understand. This is fine for hobby-level or personal projects, and I will happily use this translation to play the game in the knowledge that it might not be accurate. For anything professional, quite appropriately, a professional translator is called for.

A sample of the translated script in game can be seen in the video below. A few obvious issues, like broken speaker detection and lack of text breaking, were left unfixed to demonstrate that beyond replacing the script, there will typically be fixes required to get a nicely functioning fan translation. I reckon Claude and I can crack those without too much trouble though 😎

Although it is not the right way to translate a videogame, this is adequate for my purposes. It will allow me to achieve one of my life's two dreams - completing the Infinity series - so that I may return my full focus to the other: becoming moderately good at the piano.

Overall, an excellent result.

(P.S. see the addendum for progress on fixing issues)

Slides (63): PDF

Addendum - Fixing Issues

Since the talk, Claude and I have started tackling the remaining issues with the translation. I'll keep this section up to date with progress.

Speaker detection

As you'll notice in the video, "speaker detection" is broken in the original translation. The speaker indication appears to be part of script messages (i.e. is not controlled by opcodes/script logic), and is included at the start of messages by placing the speaker name within special brackets. For example:

【Renmaru】"No good here..."

In our early translation attempt, these messages appear with the speaker indication inline in the messagebox. It's wrong, and a bit distracting!

My original hypothesis was that we were missing a space between the special end bracket and our quote mark, which was preventing the speaker name from being lifted to its rightful place. Of course, Claude agreed (it always does), and various fruitless experiments with characters and spacing yielded little success. Claude was convinced that our translation pipeline was breaking the detection by messing with characters, and unfortunately, I had to get my own hands dirty - operating a hex editor myself, like an animal - to prove that incorrect.

After demonstrating that speaker detection worked if we left original Japanese names in the script, I adjusted the hypothesis to be that there must be some reference data elsewhere in the game files, with speakers defined. Tasking out several parallel agents to find references to speaker names in non-core script files found that there was a long list of speaker names in the DATA.BIN file. As an aside, tasking out many parallel agents to search for patterns, or even to try performing the same reverse engineering work analysis on different files, is a really effective way to increase the scope of findings within a fixed period of time. It did not take too much more fiddling to implement a speaker reference data replacement step into our translation pipeline, and now speaker detection works as expected!

Next up - text wrapping. To quote a timeless phrase:

This story is not yet at an end, the truth is not revealed - It is an infinity loop!

Everything is as it should be 🏝️💻 . . .

...

...

...

Or is it? In this post we'll learn how to identify and replace some of the pesky intel binaries that sit between us and a trip to csrutil disable to remove Rosetta for good^, and speed up iOS publishes along the way.

an al-arm-ing discovery

You can follow along if you're on an M1, or just take my word for it:

Open Activity Monitor, sort the process list by Kind.
If you don't have the Kind column, you should 😤
(you can turn it on by right-clicking the column headers)

Open Terminal and get yourself to a dotnet ios/maui project on your machine somewhere.
If you don't have one, dotnet new maui -o gathering_intel && cd gathering_intel will get you set up with a starter project

Then kick off a publish
dotnet publish -c:Release -f:net7.0-ios -r:ios-arm64 -p:EnableCodeSigning=false -v:n

Before long, you'll start to see the mono-aot-cross invocations fill the terminal. They start off like this:

Tool /usr/local/share/dotnet/packs/Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.ios-arm64/7.0.3/Sdk/../tools/mono-aot-cross execution started with arguments: ...

Take note of the path to mono-aot-cross for later. Now switch back to Activity Monitor, and try not to audibly gasp.

Just to be sure:

find /usr/local/share/dotnet/packs/Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.ios-arm64/7.0.3/Sdk/../tools/ | grep cross/ios-arm64/ | xargs file | grep executable

/usr/local/share/dotnet/packs/Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.ios-arm64/7.0.3/Sdk/../tools/llc:                               Mach-O 64-bit executable x86_64
/usr/local/share/dotnet/packs/Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.ios-arm64/7.0.3/Sdk/../tools/opt:                               Mach-O 64-bit executable x86_64
/usr/local/share/dotnet/packs/Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.ios-arm64/7.0.3/Sdk/../tools/mono-aot-cross:                    Mach-O 64-bit executable x86_64

Yes it's true: even in our arm64 dotnet install, mono-aot-cross, llc and opt - the bits that handle AOT compilation and optimisation - currently ship as x86_64 binaries and are run under Rosetta.

eh, this only affects publishing, it's no big deal!

That's fair - most of the time, we don't care that much about how long release builds take. Because of changes to the build approach in dotnet ios, or maybe just because of most everything else being arm64 on apple silicon, the development-time experience is pretty zippy (I still make heavy use of tbc though).

But what about when you're gearing up for release and start to focus on bundle size, or things like startup performance? That's the thing: The only way to know the true impact of a change with respect to bundle size or performance is to perform a release build. So at some point in your project you might just find yourself in an 'inner-dev-loop' of release builds, and at that time, build times might matter.

I went in search and found that unsurprisingly, the dotnet team already identified this gap in the arm64 binaries, and this issue tracked it. The scope of that issue was eventually narrowed and the remainder (including macos arm64) is tracked here. That means that this should eventually get resolved, maybe even in a future net8 preview, and you could just wait for that. But what if you're doing size/performance optimisation work NOW? You'll have to get your hands dirty, but is possible to solve this for yourself (for some definition of solve).

how much faster is using native AOT binaries over rosetta

So you can decide whether it's worth doing this, I've run some highly un-scientific speed tests. Here's a chart of my findings:

I tried four projects - dotnet new ios, dotnet new maui, eshop mobile client (from here) and one of my own. I added -clp:PerformanceSummary to the publish invocation to get the timings for the AOTCompile task.

On my machine, the aot compilation time reduction ranged from 30-35% across the projects - let's call it a third. Apple says the M2 gives up to 20% faster CPU performance than the M1, so if like me you have an M1 and sometimes have irresponsible thoughts about an M2, this basically saves you six to eight thousand australian dollarydoos.

how to (high level)

We saw in the github issue that the dotnet team ran into issues doing this - how can we expect to be able to make it work? We can make it work because we have simpler goals. The dotnet team has to worry about pesky things like "passing build pipelines", "architectures other than arm64", "solutions that don't just work on one person's machine" and other realities of shipping an sdk and runtime. We don't need to concern ourselves with those kinds of hassles.

We just want to take our arm64 mac and produce arm64 aot compiler binaries that aot for ios-arm64, and then somehow have them be used by the build. For that, we can build our own out of dotnet/runtime, and then just overwrite the the intel binaries we originally got from official sources with our bootleg ones. What could possibly go wrong?

Just like back in the day when we were rolling our own reflection-emit-enabled Xamarin.iOS versions, it goes without saying that you should exercise caution when replacing core parts of the dotnet build pipeline with custom built tools. It's true that we are building off tagged ("blessed") commits, but the reality is that arm64 aot compiling binaries aren't officially produced right now and the use case may not have been through the same testing rigour that supported use cases have. I haven't had any issues (yet?), but it's probably best to limit use of these binaries use to the aforementioned 'inner-release-loop' scenarios only, and use the official binaries for builds you actually want to ship. No warranties provided, proceed at your own risk, etc. etc.

how to (in detail)

With disclaimers out of the way, if you're still on board we're ready to start making a mess. With any luck, this process should only take 10-20 minutes.

First, clone dotnet/runtime:

git clone https://github.com/dotnet/runtime.git && cd runtime

Then, check out the tag that matches the version of the sdk you're using to build. You can see it in the path of the aot invocation from earlier. In this post, the invocation was:

Tool /usr/local/share/dotnet/packs/Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.ios-arm64/7.0.3/Sdk/../tools/mono-aot-cross execution started with arguments: ...

so we want 7.0.3. In dotnet/runtime, the version tags are preceded by a 'v', so:

git checkout v7.0.3

(It's important to build off the tag matching the version of the dotnet sdk you're using. If not, you may run into errors due to differences between versions. For example, you can't build off the tip of main, which right now is .net8, then use the outputs with a .net7 sdk; things will go badly. An implication of this is that when you update dotnet, or if you have different projects pinned to different versions of dotnet, you'll likely need to need to follow these steps and build binaries for each of them individually. Basically let's just hope arm64 binaries start shipping soon)

Building this repo requires certain dependencies to be available on your system. You can run the below from the repo root (where you should already be) to get them, assuming you already have Homebrew:

brew bundle --no-lock --file eng/Brewfile

Ok, now we're ready to build things. There are flags you can pass to the runtime build script to isolate the build of the AOT cross compiler, but I didn't get great results with various combinations of these (either only some binaries came out arm64, or the build just didn't work - which is maybe what the updated issue tracks). So let's keep it simple:

./build.sh -s mono+libs -os ios -arch arm64 -c Release

This should take somewhere between 5-10 minutes, and complete without issues. It's pretty impressive really (go look in artifacts to see all the things we built with one command and no shenanigans).

Now make sure that we got what we wanted:

find . | grep cross/ios-arm64/ | xargs file

You should see:

./artifacts/bin/mono/iOS.arm64.Release/cross/ios-arm64/llc:            Mach-O 64-bit executable arm64
./artifacts/bin/mono/iOS.arm64.Release/cross/ios-arm64/opt:            Mach-O 64-bit executable arm64
./artifacts/bin/mono/iOS.arm64.Release/cross/ios-arm64/mono-aot-cross: Mach-O 64-bit executable arm64

Yes! arm64 all the things!

All that's left to do is to overwrite the official binaries with our own ones. Once again, the invocation from earlier tells us where these need to go. Just in case, let's keep a copy of the original bits around (also useful if you want to do comparisons).

(Remember to substitute the 7.0.3s here and below for your version if necessary)

sudo cp -R /usr/local/share/dotnet/packs/Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.ios-arm64/7.0.3/Sdk/../tools/ /usr/local/share/dotnet/packs/Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.ios-arm64/7.0.3/Sdk/../tools/backup

That put all the original binaries under a subdirectory called backup. Now copy our new files over.

sudo cp artifacts/bin/mono/iOS.arm64.Release/cross/ios-arm64/* /usr/local/share/dotnet/packs/Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.ios-arm64/7.0.3/Sdk/../tools/

And that's it! Let's run another publish and see how it goes.

Now we're cooking with charcoal. Enjoy your 33% faster builds!
^(Only one intel binary left!)
(it's m l a u n c h)

bonus thoughts: other factors affecting build time

Switching from x64 to arm64 binaries is a nice 'free' build time improvement. There are a couple of other things that you can look at.

💡 Linking/Trimming

The less code you have, the less code needs to be AOT compiled. Using the linker will reduce the time spent in AOTCompile (and the output binary size). Some of it will be moved to the ILLink task, but the net effect should be a faster build and a happier user.

💡 Dealing with AOT-unfriendly assemblies

In the chart from earlier, "my app"'s AOT time went from ~120s to ~80s when switching to arm64 binaries. But when I first started looking at the build time, the non-arm64 AOT time was around 800s 🤯. Watching CPU usage and looking at build output made it clear - one assembly in the project took several times longer to AOT than all of the others.

The way the AOT step works is that the build system basically spawns an AOT process per assembly, for all assemblies at once, and lets the operating system manage their resource allocation. That's why in the screenshots of Activity Monitor in this post, you see a large number of processes using a small fraction of a cpu core - there are some 100+ processes trying to get their slice of 10 cores. Each of the AOT processes appears to operate on a single thread, which is fine in the beginning when there are more processes than cores and the cpu is oversubscribed. But if a single process takes much longer than the others, eventually it will be left running on it's own on a single thread, which is not very optimal. Scraping the output, I was able to see this behaviour in my own build (names removed to protect the innocent):

(n.b. Because of the probably indeterminate nature of oversubscription, it's not truly fair to compare any of the specific numbers in the above diagram, but for general magnitudes we can use it)

Essentially, the AOT of one assembly is responsible for blowing out the build time by 10+minutes. In my case, the functionality being used in that library was something that could be replicated natively without too much hassle, so I switched to that and removed the assembly. Another option would have likely been to link aggressively on that assembly to remove more of the code causing the AOT work (my guess - heavy use of generics).

how did I produce this? Probably you can do something smart with binlogs, but I just scraped the msbuild output. First, log the build to a file by adding -flp:v=diag -flp:logfile=mylog.log to your build arguments. Then, use this gist to process the file.

💡 Opting to interpret some assemblies

This is more of a build size vs performance tip, and that should be your driving factor for this (not release build time), but I'll include it anyway. For a while now, we've had access to the interpreter option which enables various scenarios. In new dotnet ios, having it enabled is currently something you probably need to do because it is easy to unintentionally trigger code-gen (my theory is we had some special BCL assemblies in Xamarin days that avoided code-gen in certain methods but now that we share with dotnet you can hit it more easily). But don't just enable it and interpret everything!

Don't: (enable the interpreter and interpret all our assemblies)

<UseInterpreter>true</UseInterpreter>  

Do: (enable the interpreter and interpret none of our assemblies, but be ready to interpret any codegen)

<UseInterpreter>true</UseInterpreter>  
<MTouchInterpreter>-all</MTouchInterpreter>  

Doing the first one will skip AOTing everything, so I guess in the spirit of this blogpost it's going to make your release builds super fast and your outputs super small, but it's also going to make things much slower.

Consider: (enable the interpreter and interpret specific assemblies)

<UseInterpreter>true</UseInterpreter>  
<MTouchInterpreter>-all,AssemblyToNotAOT1,AssemblyToNotAOT2</MTouchInterpreter>  

Here we name AssemblyToNotAOT1 and AssemblyToNotAOT2 as assemblies that will be interpreted at run-time, so they won't be AOT-compiled at build time. This will reduce output size and release build time

💡 Getting an M2

No no... just do some of the above 😎

By combining the use of arm64-specific binaries and maybe a few build tips, hopefully you can see an improvement in your release-build times like I did, plus better battery life and an improvement in your general health and wellbeing.

Finally, everything as it should be 🏝️💻 . . .

In general we are performance sensitve on Android, so the cost of JIT'ing many methods being called at startup for the first time is undesirable. Unfortunately, depending on marketplace, we are also often binary size sensitive too, making the size increase incurred by using AOT compliation also undesirable. Ugh - we can't win! If only there were a middle ground - some way to get the best of both worlds. If we could balance some of the startup improvements of an AOT'd build with the binary leanness of a JIT'd build, that might be an ideal compromise.

As you may know, a possibility of this nature has existed for a few months now in the form of a feature sometimes referred to as 'Startup Tracing' and othertimes referred to as 'Profiled AOT'. At the time of introduction, these names were technically correct, but practically less so. Where ideally the feature would be tailored to your specific app, in these early versions the 'startup' that had been traced was not. Instead, it was that of a secret, generic, Xamarin.Forms-based app stored somewhere deep in Xamarin laboratories, forced to repeatedly start up and shut down while engineers meticulously documented the methods being called in order to decide what parts of an app should be AOT'd to improve startup times.

Well that's what I heard anyway.

With the release of Visual Studio 2019 16.5 Preview 2, the Startup Tracing/Profiled AOT feature has been enhanced, and now allows developers to collect and use their own custom AOT profiles. This means that an AOT profile can be tailored specifically to an individual app - covering all the libraries and frameworks being used during startup or otherwise.

AOT Profiles? Huh?

So what's an AOT profile? Why is it called Profiled AOT? How is it different to full AOT?

Essentially, if you build a Xamarin.Android app using profiled AOT it comes out as a kind of hybrid JIT/AOT entity, containing both a mixture of ordinary .NET IL that will be JIT'd at runtime, plus some fully AOT'd code generated during the build process. Whether or not a given method in the app is AOT'd is determined by an AOT profile, which at its core is a list of methods that should be AOT'd. In theory, if a method isn't listed in the profile, it won't be AOT'd and will be JIT'd at runtime instead (although sometimes I saw results that suggest inputs other than the profile might influence whether something gets AOT'd). With that in mind, using profiled AOT makes it possible to selectively AOT-compile performance-sensitive parts of an app, allowing targeted size/performance tradeoffs and a 'best of both worlds experience'. Yes!

What about 'Startup Tracing'?

Well (in my opinion) 'Profiled AOT' is the correct name for the feature just described - using an AOT profile to selectively AOT-compile parts of the app. Startup Tracing could reasonably refer to the process of recording method calls made during startup in order to produce an AOT profile tailored towards improved startup performance. Whilst this is the most likely use for profiled AOT on Android, it's worth mentioning that an AOT profile need not include only methods called during startup, and could conceivably include any combination of methods you think should be AOT'd because they are performance sensitive.

Getting started with Profiled AOT

Making use of profiled AOT involves two distinct steps:

1. Creating (or capturing) an AOT profile
2. Use the AOT profile when building the app

The steps are outlined briefly in the documentation here, so this will be a more hand-holding version of that. Note that you'll need at least Visual Studio 2019 16.5 Preview 2 or VSMac 2019 8.5 Preview 2 to use this feature, and that the instructions here are for Windows but should be adaptable for macOS without too much trouble.

Step 1: Capturing an AOT profile

Given a general understanding of how your app starts up and the libraries it uses, it would be possible to try to create an AOT profile that covers startup by hand. However, doing so would be laborious, error prone and likely to contain both omissions or unneccessary inclusions. With VS 16.5 Preview 2, Xamarin.Android includes a new msbuild target that can build an app in profiling mode, allowing the app to automatically keep track of method invocations being made while the app is running. To take advantage of this target, you should have an Android device plugged in to your machine, and a VS2019 Preview Developer Command Prompt open to the directory in which your Android project lives. For later steps, it's useful to open the prompt as Administrator.

The name of the target for profiling is BuildAndStartAotProfiling and you can invoke it using the following syntax:

msbuild /t:BuildAndStartAotProfiling  

You'll see a lot of ordinary build output and then a few new bits. The app will also be launched on the device.

When built and launched via the BuildAndStartAotProfiling target, the app is automatically keeping track of method invocations, and (by default) is listening on port 9999 for something to connect and retrieve the logs. Although we are talking about startup, it's worth keeping in mind that when running under this mode the app records all invocations that occur, not just those on the startup path. This means that if you continue to interact with the app after it starts up, the methods that are invoked as a result of those interactions will also be captured in the logs, and will form part of the final AOT profile.

Once you're happy that the app has completed startup, you need a way to retrieve the profile data from the device. To do so, you need to keep the app running and use a second target, FinishAotProfiling, to connect and retrieve the logs.

msbuild /t:FinishAotProfiling MyApp.csproj  

That target appears to wrap a default invocation of aprofutil (on windows, lives in C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\MSBuild\Xamarin\Android\), and you'll get better diagnostics by invoking it directly:

aprofutil -s -v -f -p 9999 -o "custom.aprof"  

One point to note with the aprofutil is that it appears to require Adminstrator privileges to execute correctly (that's why I said to open the prompt as Admin earlier). However, it also requires adb to be on the PATH, which generally isn't the case for the VS Developer Command prompts. Because I'm lazy, my workaround was to open an Android ADB prompt from Visual Studio Tools -> Android -> Android ADB Prompt, then add the PATH from that prompt to the end of the PATH in the running Admin prompt. Don't @ me.

From the log you can see that a new file was written - custom.aprof. Congratulations - you've now generated a custom AOT profile!

Step 2: Using the AOT profile when building the app

The UI support in Visual Studio for working with custom AOT profiles is not complete, so it's easiest to just edit your .csproj by hand. First, you need to add a reference to the custom profile with the appropriate item type:

<ItemGroup>  
 <AndroidAotProfile Include="$(MSBuildThisFileDirectory)custom.aprof" />
</ItemGroup>  

Then, in the Release section of your csproj, add properties that instruct profiled AOT to be used, and for the default profile to not be used.

<AndroidEnableProfiledAot>true</AndroidEnableProfiledAot>  
<AndroidUseDefaultAotProfile>false</AndroidUseDefaultAotProfile>  

If you're in a project that has previously used AOT or startup tracing, you may also need to remove other configuration elements related to those.

And that's it! Now, when you perform a Release build of your app, the custom AOT profile will be used to determine which methods to AOT. You'll see slightly different output in the build log demonstrating this:

When comparing the size of AOT content to an apk that has been fully AOT'd, one that uses the startup profile should be substantially smaller. It turns out the perfect apk does exist, and in the case of the PrismZero app, the size impact is quite low:

Of course, given PrismZero is a demo app, your mileage may vary and you should run the numbers on your own apps.

What is a profile anyway?

If you're curious, you can use the aprofutil to perform basic inspection of a custom profile:

aprofutil -as custom.aprof  

This will print all (-a) kinds of entries in the profile - modules, types and methods to the console, then print the summary (-s) we saw earlier. There are a few other arguments that can be used to filter the output, but it's relatively unweildy.

It's also possible to programatically inspect the profile by referencing Mono.Profiler.Log (on Windows, it lives at C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\MSBuild\Xamarin\Android\Mono.Profiler.Log.dll) and loading the profile using the ReadAllData method on the ProfileReader class. In this format, we can see it's a set of Module (essentially, 'assemblies'), Type and Method records:

The modules, types and methods are listed in the order they are accessed, which can interesting information in itself. Because there are so many entries, I filtered most out to give an idea of what each looks like:

As we can see from this filtered set of types, during the early stages of startup there are a lot of constructors (ctor) being invoked, as well as the important Prism app CreateContainerExtension method. From the chart in my earlier post on DryIocZero (shown below), we saw that the container setup time (whether using zero or otherwise) improves substantially on Android under AOT, so it is beneficial for us to include it in the AOT profile.

Of course, being specific to our app, methods related to container initialisation would not be AOT'd using the default Startup Tracing profile in earlier versions of Visual Studio. It's the ability to produce a profile specific to your app's startup and libraries that makes this new iteration of the feature so compelling.

If you noticed earlier, we used a ProfileReader to load the AOT profile data. There's also a ProfileWriter class that can be used to write a new or changed profile back. The ProfileData class is immutable, meaning you should create a new instance with a different set of data, but we can also remember that in .NET there is no spoon, and just alter an existing instance directly using reflection.

Modifying a profile can open up more advanced scenarios. For example, we could merge the contents of two profiles (e.g. a 'logged out' startup path and a 'logged in' startup path), or we could combine information from a startup trace with other convention-based AOT decisions (e.g. AOT all startup methods + all ContentPage InitializeComponent methods), if we thought that were appropriate from a size/performance tradeoff perspective. Whilst regenerating a startup trace during CI is more challenging because it requires the app to be launched, regenerating convention-based profile contents for a Forms-based app at build time is quite reasonable. Maybe an experiment for a rainy day.

Should I use this?

Unless you are extremely binary-size sensitive, profiled AOT really does look like a compelling feature to start using. It's relatively easy to set up, and should net some good performance improvements if you aren't using full AOT now, or a nice size decrease without a major performance impact if you were already using full AOT. Keep in mind that as an app evolves, the methods called during startup may change too, so regenerating a profile from time to time is a good idea.

Happy profiling!

(This is a pretty long and dense post, so make sure you're comfy!)

Does DI container choice even matter?

It does, because of the overhead that's involved. How much, and when it is incurred (registration/resolve), does vary from container to container, as demonstrated in these benchmarks. Note that these are times for 500,000 resolutions, so we should take care when drawing conclusions for our purposes.

Working in mobile - a comparatively performance-sensitive environment - Xamarin developers hopefully gravitate towards containers with lower overheads. I like DryIoc, and admire the commitment the author @dadhi places on keeping it light and fast, without compromising on features.

Inspired by Clinton Rocksmith's recent post on using Lazy<T> to defer subgraph resolution, I decided to cross off something that's been on the list for a couple of years now - investigating DryIocZero, a sibling to DryIoC that allows you to create a container that incurs zero registration overhead at startup.

Zero calories ʳᵉᵍᶦˢᵗʳᵃᵗᶦᵒⁿ ᵒᵛᵉʳʰᵉᵃᵈ? ɴᴏ ᴡᴀʏ ᴛʜᴀᴛ's ᴘᴏssɪʙʟᴇ

I mean, that's what they said about Coke Zero too.
(mum tells me there's a catch)

It does sound too good to be true. So what's the deal? From the DryIoc extensions page, DryIocZero is "[a] slim IoC Container based on service factory delegates generated at compile-time by DryIoc".

In short, you use the DryIoc library at compile time to generate code that describes a complete, configured and validated implementation of your container. The result is a class that you can new up that is immediately ready to resolve the roots you specified - there's no need to burn time during startup adding registrations, and no need for DryIoc to process types and understand the dependency graph or resolution method for these requests. A simple example should make the idea clearer:

On the left is a basic dependency graph with two roots A and B (viewmodels, maybe) and a small number of interface dependencies. Below that, is some straightforward DryIocZero registration code - the same code you'd write when working with DryIoc normally. On the right is the relevant code that DryIocZero generates for that configuration, which I have tided a little for readability.

Effectively, each resolution root gets a dedicated and self-contained implementation that is scope aware, generated based on the container configuration. When a root needs to be resolved, ResolveGenerated calls the appropriate method and you get your requested object. As you can see, resolution implementations involve no interface indirection, and no reflection - both of these are desirable properties in a Xamarin world. The Container class (which includes more code in a static partial counterpart) can be instantiated directly with no need for further configuration. Sounds pretty good!

Proof of Concept - Prism + DryIocZero

To put this approach through its paces with a more thorough test, I decided to try to configure a Prism app using DryIocZero. Though I haven't yet used Prism in a real app, I have the impression that it is both opinionated and configurable enough that integrating DryIocZero should need thought, but be possible.

In the end, the Prism source, plus a good post from @dansiegel (Using unsupported DI containers with Prism) were enough to set me on the right track. It seems to have worked out, and you can check out the final result here. I started out with the Prism template and each commit covers a specific aspect of the conversion, so you can use those to follow the process if interested. I'll call out a few bits here.

Creating Registrations

A large amount of the setup lands in the Registrations.ttinclude file. This is the T4 template file that needs to be modified to include the container configuration. In addition to convention-based registrations for services, viewmodels and pages, my implementation moves the registration of Prism core types out of startup to be performed at compile time.

Parts of the set up are moved to seperate methods, so you can get a better idea of the high level flow: The first call worth noting is the LoadAssemblyWithDependencies call. It is a crude method I pulled together which attempts to read the deps.json of the app assembly in order to load other required assemblies into memory. Since DryIocZero creates a real DryIoC container from which to generate the compile-time version, any types you are registering need to be loaded into the running environment. You can reference assemblies manually in the T4 template, but I found the approach of dynamic and reflection-based type resolution to be more manageable, particularly since there's no C# intellisense anyway. This implementation makes use of the Microsoft.Extensions.DependencyModel package to load referenced nuget package assemblies into memory.

The next point of note is the RegisterPrismTypes method. This takes the registrations found in the core PrismApplicationBase class, and instead wires them up as part of the compile-time container. This approach is somewhat brittle (would need to be kept in sync with any changes the Prism team makes), but is good to take off the startup path.

In addition to registering the types here, I intercept RegisterRequiredTypes in the app subclass, to prevent Prism from also trying to register the types.

The last interesting point in the registrations is probably the convention based page/vm registration. This scans for pages and viewmodels and registers them in the container according to Prism requirements.It's mostly straightforward, but the last line is worth a mention. Each page/viewmodel combination is added to a list and an additional modification I made to the generation template results in the creation of a RegisterPageTypes() method on the container. This method can be be called at runtime to configure the Prism page registry and viewmodel locator based on registrations, again without requiring assembly scanning.

Integrating with Prism

The other major modification to fit DryIoCZero is the creation of an IContainerExtension, an abstraction that Prism operates against to allow its use with different DI frameworks. Again, Dan's post here provides a great overview. Since the container methods are very generic, most methods end up being simple argument pass-throughs to the Container implementation; to the point that we could probably just adopt IContainerExtension directly. In this case, I created a dedicated class.

In setting up the extension, I opted to completely remove the runtime DryIoC dependency, and disallow support for adding type to type registrations at runtime (delegates and instances are still supported). To my limited understanding, mutation of the container in Prism is primarily required to support the use of Prism modules - if you are not dynamically loading code or doing hot patching, you can live without supporting these kinds of registrations at runtime.

For the purposes of investigation, I did create another implementation of IContainerExtension that chained together a runtime DryIoc container and compile time DryIoCZero container, which you can see here. Using this implementation, you can combine both compile time registrations with dynamic runtime registrations if neccessary. By adding an UnknownServiceResolver rule to the runtime container, this even supports split resolutions. For example, if you dynamically load an admin module with types that depend on an existing UserService that was registered at compile time, DryIoC will resolve any new dependencies from the dynamic container and check the compile-time container for missing ones. I'm not sure of the overhead on this, but the technique is at least possible.

What's the impact?

I'm optimistic about the performance improvement that DryIoCZero can provide, especially on older devices. That said, I'm cautious about relying on the results of my benchmarks, because I'm not a pro benchmarker. I will say that these results seem consistent and are in line with the expectations that we have (i.e. that DryIocZero will be faster). These tests were run on a version of the PrismZero app with a few more services, pages, viewmodels and dependencies defined; not quite representative of a production-complexity app, but more complex than the template.

In order to appease my superstition of the possiblity of second-order effects, rather than measure only the container creation time, I opted to measure something closer to Prism app startup time (excluding Xamarin.Forms initialisation). This means starting the timer right before the built-in template calls LoadApplication, which includes instantiation of the App subclass. The timer is stopped in OnInitialisedAsync, after Prism startup and container registrations, and prior to any navigation. This means that the times do not represent the exact figures for compile time vs run time container instantiation, but - as the only thing changed between runs - give a good feel for the difference when using each. If the results are reliable, the performance impact can be substantial on older devices. My expectation is that a DryIocZero container will scale more favourably with increased number and complexity of registrations, but I have not tested this. It would be awesome to see if someone else could reproduce a similiar result independently, to give me confidence that this doesn't include anything that makes it incorrect.

No free lunches

Amazing! We should all switch to DryIocZero immediately, right? Maybe, maybe not. In reality, there are caveats, considerations and compromises that using DryIocZero involves. Before negatives, I want to run over the good bits.

Pros:

Low overhead:
Of course, this is the primary benefit. Given the results we've seen, compile time generation of the container can take a bunch of the startup impact of DI out of the picture.

Resolution performance appears to depend on whether your code is JIT'ed or AOT'ed. When JIT-ing, DryIoC resolves slightly more quickly, whilst when AOT'd, DryIoCZero comes out on top in most cases. I reproduced this behaviour both on device and on my workstation using the IoC benchmarking project, so I am confident in this assessment.

That said, given these times are in milliseconds and are measuring 500,000 resolutions, the difference between resolution times for either is probably marginal. On the slowest device when JIT'ing I saw resolves take 2-3 ms longer using DryIocZero. For saving 500ms+ at startup, I consider that a fair trade.

No penalty for convention-based registrations:
This is kind of part of the previous point, but worth an additional callout. For convenience, it's common to prefer to create registrations based on some kind of convention, or based on the presence of marker types. You can see this technique in use in the Prism demo app.

Writing code to register types based on conventions like this is nice, because we don't have to go back to the container and add new registrations every time we create new services etc. However, doing so requires reflecting over assemblies to find the matching types, which adds to startup time. With DryIocZero, this work gets done at compile time, so we get the convenience of convention based registration without addition cost.

Less 'magic':
Because DryIocZero generates real code into your project, there's less mystery involved in injection. As the developer, you can set breakpoints inside resolution methods if you want to step through the process. The compiler gets to work on the resolution code as well, so it is eligible for AOT and optimisation. The linker is also better informed because of the lack of indirection. For example, it can see exactly which constructors are being invoked on types, and won't link them out. In the case of convention based registrations, where eligible types would normally only be detected at runtime, the linker again gets the benefit of seeing the types referenced in the container at compile time.

Extensible:
Probably both a pro and a con of DryIocZero is its delivery method - a set of classes you add to your project, including a T4 template. Since the template itself can be modified you can add your own code generation functionality. An example of this is the automated generation of RegisterPageTypes() in the demo app.

Cons:

Unfortunately. it's not all sunshine and rainbows 😭. There's no denying that there's more effort involved in setting up and maintaining injection via DryIocZero, for a variety of reasons. Installation is more involved than your standard NuGet (but the instructions are easy to follow). Some other considerations have reasonable solutions or workarounds that I worked out quickly at the start, others could pose ongoing development time overhead.

As I look to try using DryIocZero on a new project, I'll find out whether it's truly viable. I'm think that I probably have a higher 'inconvenience' threshold than many (but hey because of that I've been hot reloading code forever thanks Frank). For now, here are some pain points that I found.

No access to runtime state in registrations:
A well documented limitation of Zero is that you can't preconfigure implementations that depend on runtime state. Since the container is effectively created ahead of time, this makes sense, and rules out the use of methods like RegisterInstance(...) and RegisterDelegate(c => ...).

So what if you have a dependency with an implementation that can't be known until runtime? In situations like that, you can use the RegisterPlaceholder method to tell DryIocZero that you will provide an implementation at runtime. That allows DryIocZero to construct resolution implementations with (appropriately) placeholders for the missing types, allowing the container to compile. As an example, here is the code generated in a resolution if we specify that the IamNested dependency from the beginning example will be provided at runtime:

If we don't register a placeholder at all, DryIocZero will not generate any roots that involve graphs with that dependency - how could they compile? In the case of our example, this would remove B from the generated code - and write an error at the top of the container like the below:

Although it's good the the container is validated, I actually tend to think that a missing dependency should cause the entire container to fail generation. In the above case, since the container code still compiles, you could end up hitting the missing dependency at run time.

Semi-mutable implications:
I'm coining the term 'semi-mutable' to describe a DryIocZero container's mutability. A container generated by Zero is actually independent and self-contained - although the generation process uses DryIoc, the output is lighter-weight and has no such dependency. There are various implications to this, but one to keep in mind is that it is not an IContainer so can't be used in place of one, and does not support everything an IContainer supports. Looking at this issue, this is something that might change in the future, but it is this way for now.

We've seen that a DryIoCZero container supports some mutation - specifically, the ability to replace placeholders, or add new registrations with instance or delegate implementations. However, it doesn't support the 'standard' registration syntax:

// this overload does not exist on a zero-generated container
container.Register<IThing, Thing>();  

If you think about it, this is reasonable. We're telling the container that it needs to be able to later resolve IThing to a Thing, which may have arbitrary dependencies and its own resolution strategy - but this isn't a DryIoc.Container that knows how to work out things like that. This is lightweight, purpose-built container class designed to resolve exactly what we told it to be able to resolve at build time.

For most purposes, this probably isn't a big deal. However, in an interpreter powered, hot-patch-friendly world, you might need a fall back. One option is to provide your own delegate that reflects over Thing and attempts to resolve the dependencies from the container. It won't be blazing fast, but given that you've already accepted the perf hit of your new dynamic code being interpreted (on iOS, at least), this might be acceptable. Another option is to chain the DryIocZero container to a fully fledged DryIoC container, as I mentioned when talking about the Prism demo.

Configuring the configuring of the container:
Again, a strength and weakness of DryIocZero is its implementation via T4 templating. It's a clever and workable solution, but T4 templates are a bit awkward to write code in. For that reason, I'm tending towards favouring convention-based registrations - the idea being that if you get the conventions set up properly at the start, you'll rarely need to revisit it.

The obvious challenge is the lack of C# autocompletion. Since we're typing into a T4 template that's to be expected, but it means working out overloads and namespaces can be harder.

The bigger challenge is making sure that the environment that's executing the template has access to all the required types to be registered. You can add references to the T4 template to have assemblies loaded, and the first one you'll need to add is a reference to your own app. That's easy enough thanks to a few available substitutions. I don't love it but I can live with it:

With that reference added (and your project built), you can now refer to your types. However, if you need to register something from another assembly, or a type of yours that derives from a type defined in another assembly, like a Xamarin.Forms ContentPage subclass, you'll need to reference that assembly too. One way to handle this is to keep adding references to the T4 template as you find that you're missing them, but that isn't much fun. An alternative approach is to dynamically load the dependencies of your app before configuring the container, and use reflection-based techniques to reference types. It sounds bad, but since we already aren't getting autocomplete and we're favouring convention-based registration I think it's the better approach. This is the technique I used in the Prism proof of concept.

🐔 Which came first 🥚:
As shown earlier, the T4 template used by DryIocZero needs to use the types from your project to know how to generate the container implementation. That means that your project already needs to be built to generate the container. Of course, once the container is generated, your project needs to be rebuilt to make use of the generated container implementation. Aaahhh.

In reality, the way DryIocZero is implemented means that it avoids a true chicken-and-egg problem, because the generated code goes into a partial class. Since the other part of the partial class is always in the project, there's no problem in having your project reference the container even when the resolution implementations haven't been generated yet.

However, changes to your project can cause the container to fall out of sync. For example, adding a new dependency to an existing registered type will break the generated container, as it will be missing the new constructor argument. To fix this, we just need to regenerate the container. Unfortunately, to regenerate the container we need to build the project with the updated type dependencies, and we can't build the project because the existing generated container is missing the construc-- Aaaaaaaaaaaahhh.

The answer is to just delete the generated container before trying to build, but this can be a bit arduous and awkward. I feel like with some clever project structuring this could be improved, which might work in with the next point.

Platform dependencies:
This is probably the major outstanding question I have around putting together a reasonable implementation of DryIocZero in a Xamarin app. Typically, cross-platform apps define their DI container in the shared project, with a mechanism that allows the platform heads to perform any platform-specific registrations for implementations that exist in the heads only. This can be as simple as providing an Action<Container> to the shared project during initialisation, or using something like Prism's PlatformInitializer pattern. How to handle this using DryIocZero?

The easiest option is to just define those dependencies as placeholders and register them at runtime. This again needs to work in with the constraints around runtime registration - constructed instances or delegates only, which implies the need to hand-resolve any dependencies. Looking over existing projects, this is probably an acceptable approach - between Xamarin.Essentials, other plugins and netstandard, many platform dependencies are already abstracted.

The better option might be to move generation of the container (now, containers) to act on the platform heads. A shared project could host the DryIocZero classes, and the template could <@ include @> a file that exists in each head with the implementation for RegisterPlatformDependencies(). It might be worth investigation.

Do the Pros outweigh the Cons?

Whew, that was a lot of words.

I do see several challenges to using DryIoC in anger in its current form, but I think the answer comes down to the project, team and tolerance for straying from the beaten path. There are other aspects I can think of but haven't investigated yet, such as working this into a build pipeline and how it might work in a team and with PRs. My spidey sense says that the container should probably not be checked in, and should be generated at build time.

Personally, I'm game to give a try, and will be using in on my next project. If it's not working out, falling back to DryIoc proper involves little more than copying the configuration code out of the T4 template and into the app, making it a relatively low-risk opportunity. If people are interested, I'll keep them up to date with how it goes!

This meetup itself was streamed on Twitch, so you can also watch the full thing back at your leisure (~1h 15m):

Slides are available at the end of the post, and a little detail on each of the new areas from this talk is below.

Multi Window

WWDC Reference: Introducing Multiple Windows on iPad

On iPad, iOS13 (strictly, 'iPadOS') adds the ability for individual apps to work with more than one window.

There are various ways that a new window can be created - from the dock, from a shortcut item, via a drag and drop operation, or programmatically.

The positioning of an app's visible windows are fixed to a few variations (split pane or floating - same as with multi-app multitasking). However, the total number of windows an app can have (that is, including backgrounded windows) is effectively unlimited.

To model multiple window architectures, iOS13 introduces a few new supporting classes; importantly the UI(Window)Scene and UISceneDelegate classes - one of each will be instantiated for every new window created by the application. The UISceneDelegate, among other things, handles the 'UI lifecycle' of the window, so if you opt in to supporting multiple windows, iOS will stop calling the UIApplicationDelegate UI lifecycle methods and instead call them on the SceneDelegate(s) of the windows involved in the changes. Whilst multiple windows can offer a lot of power and flexibility, it does come at the cost of additional complexity. For comparison, I diagrammed an indicative object instance graph in single window and multi window scenarios.

Although it's true that the second example contains three windows (so we should expect it to be more complicated), everything on the left half of the diagram is required even when a single window is used. Supporting multiple windows also comes with other considerations such as:

• Accounting for iPhones of all versions, and iPads running iOS12 and lower (no multi-window support)
• Removing any baked in assumptions about the existence of just one window
• Seperating process-level concerns and initialisation from interface level initialisation
• Cross-platform experience

Given the above, I'd personally be looking at multi-window only if I had a really strong case for it in a mobile/tablet app. On the other hand, for an iPad-focussed app - given Marzipan - it might make sense to start adopting this now.

Sign In With Apple

WWDC Reference: Introducing Sign In with Apple

Sign In With Apple (illegally abbreviated to SIWA here and in the slides) is a new OIDC compliant (with some 'peculiarities') auth mechanism that ships with iOS13. It provides a streamlined auth experience for both users and developers, by leveraging features circumstances guaranteed by the use of a signed-in Apple device.

From the user's perspective, SIWA offers a familiar, convenient and privacy-friendly signup/auth flow. No passwords are involved, and the user can choose to share or hide personal details such as their name and email address, without compromising the functionality of the application.

When the user selects the 'Hide my email address' feature, Apple provides the developer with a random-looking email address, and relays any emails sent to that address to the user's actual address. The user can disable this relay from Settings at any time.

From the developer's perspective, SIWA also offers some benefits. Apple provides buttons and sign-in UI for the process that can be invoked with a small amount of code. SIWA also provides a stable user identifier across devices and a reliable assessment of whether it thinks the user is real (as opposed to being a bot).

According to its developer policy, Apple will require apps that rely solely on social login to include SIWA as a sign up/auth option. I read somewhere that it will be enforced for existing apps from March 2020 but don't quote me on that.

CoreNFC

WWDC Reference: CoreNFC Enhancements

CoreNFC allows developers to add Near-Field Communication (NFC) capabilities to their apps when running on devices with the appropriate hardware (all iPhones since the iPhone 7, but not iPads). First introduced in iOS11, CoreNFC has traditionally offered a relatively restricted set of features to developers (when compared to that of Android), essentially limiting interactions to NDEF tag reading. With iOS13, CoreNFC supports new features that significantly increase the applications of NFC on iOS devices.

NDEF tag writing:
It's now possible to write NDEF messages to tags using the WriteNdef method on INFCNdefTag. CoreNFC provides convenience methods for constructing string and URI-based NDEF payloads via the NFCNdefPayload.CreateWellKnownTypePayload overloads. The addition of NDEF writing helps bring iOS NFC capabilities closer to parity with that of Android.

Native tag protocol implementations:
In iOS13, CoreNFC also allows developers to interact with several kinds of NFC tags via their native protocols. This includes ISO7816, ISO15693, FeliCa, and MIFARE tags. With the appropriate entitlement, and use of the new NFCTagReaderSession and NFCTagReaderSessionDelegate classes, you can now detect these tags and send commands to them using dedicated interfaces:

For demonstration purposes only and not because I am a nerd, I was most interested in trying out the MIFARE tag support, knowing that tags in Nintendo Amiibo use that technology. After consulting the MIFARE datasheet and reverse-engineered Amiibo data structure, I was able to successfully detect and read game/character identifiers from an amiibo using MIFARE commands, which could then be send to Amiibo API to perform an Amiibo lookup. Cool!

(In practice, I found the scanning to be a little bit unreliable, but I could have been holding it wrong)

Not satisfied with just identifying an amiibo, I also investigated the legally, morally, and ethically murky world of Amiibo cloning. In the image of the MIFARE datasheet above, the FAST_READ command is shown, but there is also a WRITE command that allows data to be written to a specified page. With the right kind of blank tag (NTAG215), and a mechanism for decrypting and reencrypting an Amiibo dump, it's possible to use the WRITE command to clone an Amiibo in a manner that a Nintendo Switch considers valid.

Cloning Amiibo raises so many ethical and moral questions, but more importantly, it relies on secret Nintendo key material that shouldn't be checked into the repo. Given that, I removed the cloning implementation from the demo app before checking it in, but left the high level flow in the comments for the inspired reader. It isn't too difficult (standing on the shoulders of giants), but it is a little involved.

Demo App

The "Hello iOS13" demo app has been updated to include the new demos on Github: rdavisau/hello-ios13.

Slides

Links for the slides are below.

Slides (56): PDF

Although I mostly work on Xamarin things, ML is an area of interest and I'm glad to have a good .NET-friendly framework to work with in ML.NET. Lately I have been playing with Tensorflow and ML.NET and realtime processing of video game feeds, to do things like scene detection and digit classification (score, health) etc.

My (ridiculous) @linqpad-based Smash Brothers Classic Mode tracker is coming along, now with damage recognition thanks to a custom digit classifier + @migueldeicaza's TensorFlowSharp binding.. very easy to consume a Tensorflow model from the .NET world 🤓🎮 pic.twitter.com/grDnFnqemA

— Ryan Davis (@rdavis_au) March 23, 2019

Do I need this in my life?

The answer for many people is probably 'no'. If you're a Xamarin.Forms user, you don't typically deal directly with UITableView and UICollectionView APIs, so this probably wont be of interest. Even if you work 'natively' with Xamarin.iOS, application of the MVVM pattern typically results in the delegation of datasource/control interaction to your MVVM framework of choice via bindings or dedicated helper classes. For example, ReactiveUI includes a ReactiveTableViewSource class that allows you to work with a UITableView in a more MVVM-friendly way, relieving you of the need to deal directly with nasty indexPaths and APIs like BeginUpdates. In this case, again, the addition of diffable data sources probably won't interest you.

If you do build iOS-only applications using Xamarin.iOS, like I do from time-to-time, diffable data sources do present an interesting option for table/collection view configuration.

The old way

If you've ever worked with a UITableView or UICollectionView directly from Xamarin.iOS, you're probably aware that you control the way those controls get data by passing in an instance of a 'data source', typically a subclass of UITableViewDataSource or UICollectionViewDataSource. For simplicity's sake, I'll focus on UITableView from now on, which requires a UITableViewDataSource to implement callback methods that answer three key questions:

1. How many sections does the data for this table have?
2. How many rows does a given section in the table have?
3. What cell should be displayed for a given row in the table?

Essentially, when the tableview is put on screen or reloaded, it calls the first two methods to determine the size of the dataset. Then it calls the third method as and when it needs to display cells on screen. Your implementations typically call to a backing store like a List<T> for things like the counts. When I first started out with Objective C this was a bit of a mind bender for me, but a callback-based configuration can be very flexible.

The challenge with the API then, is handling changes to the dataset. For example, inserting and removing rows, or reordering them. Changes of this nature need to first occur in the backing store (e.g. your List<T>), but then the precise set of changes need to be described to the tableview in order to have it animate the changes nicely. For example, if the second item from the list was removed, and a new one was added at the end, a set of calls like this would be made:

TableView.BeginUpdates();

// remove the second item
TableView.DeleteRows(new [] { NSIndexPath.FromRowSection(1, 0) });

// add item at the end
TableView.InsertRows(new [] { NSIndexPath.FromRowSection(5, 0) });

TableView.EndUpdates ();  

This is a simple example, but changesets can be quite complicated to describe. Your code 'describes the diff' between the current state and new state; a task that can be generalised but is not very fun. Getting it wrong results in the dreaded update assertion and crash:

*** Assertion failure in -[UITableView _endCellAnimationsWithContext:]
*** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Invalid update: invalid number of rows in section 1.  The number of rows contained in an existing section after the update (1) must be equal to the number of rows contained in that section before the update (2), plus or minus the number of rows inserted or deleted from that section (0 inserted, 0 deleted) and plus or minus the number of rows moved into or out of that section (0 moved in, 0 moved out).' 

A diff algorithm is a diff algorithm, and should 'always work' once you get it right. Still, in an async-friendly world, inconsistency can also occur if the backing store is touched during an update cycle.

A common 'resolution' for issues of this nature was to abandon Begin/EndUpdates and call ReloadData, which causes a full/expensive, non-animated, reload of the entire table view. Apparently Apple was unhappy that this kept taking place, and now we have a new option.

New shiny - the diffable data source

With the introduction of iOS13, three new interesting classes appear in UIKit/Foundation. First, we have two new data source base classes:

• UITableViewDiffableDataSource<TSectionIdentifier, TItemIdentifier>
• UICollectionViewDiffableDataSource<TSectionIdentifier, TItemIdentifier>

Additionally, there's a 'snapshot' class:

• NSDiffableDataSourceSnapshot<TSectionIdentifier, TItemIdentifier>

All three classes are generic to a section and item identifier type. The idea behind these classes is that rather than implement callbacks like NumberOfSections, and RowsInSection, we instead create and provide a snapshot of our data to a diffable data source using a new ApplySnapshot(snapshot, animatingDifferences) method, and iOS automatically determines the changes, updating the tableview contents and animating the transition for us.

Look ma! No manual diffing!

Cool right? By using a diffable data source, we save ourself the hassle of manually specifying the way changes should be animated. But how does a diffable data source know what has and hasn't changed?

You might reasonably guess that the diffing is performed by keeping track of which object instances are added or removed between snapshots. In practice, that would be too brittle - actions like refreshing data from an API typically result in the creation of an entire new set of objects - so the diffing needs to work more on a basis of semantic equivalence.

The mechanism for this ends up coming down to those two generic type arguments we saw earlier: TSectionIdentifier and TRowIdentifier. In native iOS land, these type arguments must conform to the Hashable protocol, which essentially defines the iOS equivalents of .NET's GetHashCode and Equals methods. Similar to .NET, in iOS the ultimate base class NSObject conforms to Hashable and the appropriate methods are exposed in Xamarin.iOS as GetNativeHash and IsEqual. UIKit uses these methods on the section and row types to decide whether items are being added, removed or rearranged between snapshots. So all we have to do is implement those, right?

Diffable powers without deriving from NSObject

The problem with having to implement GetNativeHash and IsEqual in Xamarin.iOS is that it requires types to be derived from NSObject, which (at least for me), is an unacceptable constraint. First and foremost, NSObject is not portable. Beyond that, deriving models from it precludes deriving them from other base types, and a need to derive from NSObject means you can't use types that you don't own, because you can't force them to derive from NSObject. What's the answer then? There are probably a few options, but the one I find least intrusive is the use of a wrapper type. The most basic implementation is one that forwards NSObject hash calls to the inner .NET type, like this:

Now, given a T that has a meaningful GetHashcode() implementation, we can pass an IdentifierType<T> to our diffable data source and snapshot methods. Another approach I like is to provide a Func<T,U> that will return the hash-friendly representation of T. That would look like this:

With the latter approach, we can use any type with diffable data sources, even if we don't own it and can't modify it to provide a meaningful GetHashCode() implementation.

Diffable powers without doubling your LOC

This still isn't that great. Wrapping the types means writing verbose, heavy handed code with a lot of type parameters all over the place. That said, it should be possible to enscapsulate that in a way that gives us a more pleasant consuming experience. I present my attempt, EasyUITableViewDiffableDataSource (name subject to change).

(It's easy to consume, not easy to read ok)

EasyUITableViewDiffableDataSource (name subject to change) exposes a much friendlier API to a C# consumer. It removes the need to think about anything NSObject-related, and lets the consumer work directly with .NET types, taking Funcs at the time of creation that configure section and row identifiers. It also exposes a new overload of the ApplySnapshot method that works directly with .NET types and handles the busywork of preparing a UIKit-friendly snapshot using our IdentifierTypes automatically. This is almost perfect, but it has one glaring issue - the type signature:

A type signature like the one above this poses two major issues. The first is verbosity and awkward usage - when you new an instance of a type like this, you have to specify each type parameter explicitly - it can't be inferred from constructor arguments.

The second is the brittleness - specifying each of the type parameters explicitly (in the constructor, and/or in a field/property definition) means lots of places to make changes if you decide your row or section identifier needs to be different. Not to worry, these concerns with EasyUITableViewDiffableDataSource (name subject to change) can be addressed.

<Diffable, powers<without<too, many>, Ts>>

The answer is a static helper that performs the creation of the data source. A static helper method is eligible for method argument type inference, which resolves the verbosity during use issue.

The brittleness can also be addressed by recognising that the existing set of generic type arguments expose what are primarily configuration/implementation details. The only interesting type argument to the consumer after the data source is created is the top level element type (TSection), which is what gets passed to the .NET friendly ApplySnapshot method. Knowing that, we can hide the gory details of the EasyUITableViewDiffableDataSource (name subject to change) behind an interface, and return the interface from our static helper method.

The helper also gives us the opportunity to address suboptimal API in the original diffable data source types, which require generic type arguments to be specified but do not make use of them when providing arguments for the getCell callback.

Was it worth it?

Whew, that was a ride. But now, a .NET friendly diffable data source can be configured very naturally:

As a reminder, that configuration gets you free tracking on every call to ApplySnapshot (same video as earlier):

The highlights from this approach:

• No type arguments specified during creation - everything is inferred

• The returned type, IUITableViewDiffableDataSource<Game> couples the code to just the core model type

• No need to derive from NSObject, or to modify existing types, or to prepare snapshots directly

• Automatic typed model in GetCell

• Automatic change tracking!

The potential downsides from this approach:

• Introduction of an unconstrained generic NSObject subclass. I am not well informed enough to decide whether that's a major issue. In an interpreter world the idea that we know all possible Ts for a generic NSObject subclass at compile time is no longer valid, so maybe it's less of a problem these days

• Additional allocations incurred during the creation of wrapper objects. Since iOS is a beast, it probably isn't a major concern. However, a modification to EasyUITableViewDiffableDataSource (name subject to change) could involve reuse of wrapper instances between snapshots.

• Supported on iOS13 and above only.. enough said

• (Probably one I should have mentioned earlier 💀) - bugs in the diffing. There are a few odd behaviours I've noticed as I stress test this with more complicated changesets, including the ability to crash the app. Given we're seeing fairly frequent updates to iOS at the moment, it might be worth waiting a little longer before going all in on this. . As it turns out, the bugs I encountered were actually in UITableView proper and not in the diffable data sources. Invoking Begin/EndUpdates directly against the UITableView with the same set of updates (as diffable does internally) causes the crash. I found this when Steve Breen (the guy in the WWDC video! :O) drove by my tweet about the bug, asked for a repro, and then verified the issue was upstream. As someone who once found themselves in a position where they were often being blamed for bugs in upstream dependencies, I am happy to see it wasn't a bug in diffable after all.

If you fall into the small niche of use cases in Xamarin.iOS that benefit from diffable data, I think that the upsides outweigh the downsides. If nothing else, the new data source types will make life easier in my demo apps and proof of concepts. If you're interested in giving it a try, the relevant classes are all available here.

Happy (not having to do) diffing!

The magazine itself includes several other interesting articles, all with a .NET focus. It has both print and online versions, the latter of which can be accessed here:

you can find my article in this issue alongside a very impressive line up of authors 😵

I have to give a special thanks to Kym Phillpotts, whose awesome Xamarin.Forms UI challenges served to demonstrate the sophistication of UI you can produce with the framework, all whilst adding a splash of colour to the article. I was also very fortunate to get a lot of help from David Ortinau, who reviewed and provided feedback on my drafts.

Dave also found a copy of the article in the wild at Microsft Ignite and tweeted to let me know. Thanks Dave!

Having conquered print, I suppose I now need to decide between film and tv next 😎

Slides are available at the end of the post, and detail on each of the new areas is below. If you want to play along, you can install the Xamarin.iOS / Xcode11 Preview and clone the sample app.

Dark Mode

(WWDC Reference: Implementing Dark Mode on iOS)
As you have probably heard, iOS13 ships with a new 'dark mode', in the form of a system-wide theme setting that is observed by all Apple apps and various aspects of UIKit. This themeability is mostly enabled by new dynamic behaviour in UIColor and in image assets - specifically, the ability for these to take on different appearances depending on whether dark mode is enabled or not. Apple's semantic colours (e.g. UIColor.LabelColor, UIColor.DarkTextColour) are all dynamic in iOS13, which means they automatically change based on the user's selection. A number of new semantic colours have been added in iOS13, some of which you can see below.

You can easily determine which colours on the UIColor class are dynamic, as they have the -Color suffix. For example, UIColor.SystemGreenColor is dynamic, whereas UIColor.Green is not.

By targeting iOS13, your app also opts in to themeability, which means that parts of it will begin responding automatically to changes - the parts that were already referring to semantic colours, either explicitly in your code, or by default (e.g. a label that hasn't had its text colour modified). It's likely that an existing app will not be using dynamic colours throughout, so there may be not-insubstantial effort involved in making it work cleanly in both modes. Although Apple talks a lot about defining dynamic colours in your asset catalog, dynamic colours can also be defined programmatically, which may simplify migrations depending on your apps architecture. This is achieved by creating a UIColor using a new constructor that takes an Action<UITraitCollection> that iOS will call whenever the theme changes; in this action you can test for the current theme and return the appropriate colour. It sounds complicated but is simple in practice:

In iOS13 Dark Mode can be toggled by the user (relatively) easily via Control Center, so your app should be ready to have the theme changed while it is running - it's not enough to check the setting at startup. If you need to perform arbitrary work on a screen in response to a theme change, you can do so by overriding the TraitCollectionDidChange method on UIViewController (which could be your base viewcontroller or Xamarin.Forms PageRenderer), and making changes based on the new theme setting. For example, in the below demo (which represents me and my feelings on non-dark mode apps) we replace images, change text, and toggle animations when notified that the theme has changed in TraitCollectionDidChange.

If you want to target iOS13 but aren't up to making the changes for theme support, you can opt your app out of dynamic behaviour by setting the OverrideUserInterfaceStyle on a view-like element to UIUserInterfaceStyle.Light or UIUserInterfaceStyle.Dark. Your UIWindow is a good place if you want it to apply to the entire app. Make sure to CheckSystemVersion before you do, as the selector is not present on devices below iOS13.

Currently Apple is not requiring apps to support dark mode, but as the OS and built in apps support it, it will be interesting to see whether users expect it. When updating to iOS13 or setting up a new iOS13 device, users are explicitly asked to select between light and dark theme, so they will be aware of the feature.

In the Xamarin.Forms space, Gerald has put together informing and entertaining proposal/specification for handling the new appearance considerations for iOS here. It also begins to consider the equivalent theming coming in Android 10.

PencilKit

(WWDC Reference: Introducing PencilKit)
PencilKit is a high performance drawing framework that makes it super easy to give users of your app a sophisticated but familar drawing environment, with a very small amount of code. As the name suggests, the framework is optimised for use with Apple Pencil (which is quite the feat of engineering - watch the WWDC video for more information), but also works fine for users providing direct touch input. Two core pieces of PencilKit that you're likely to interact with are PKToolPicker and PKCanvasView.

PKToolPicker

The PKToolPicker class lets you control the display of a toolbox UI for drawing tasks.

You might think the UI looks familiar - that's because it's made up of the same core set of tools that you have access to for markup in other Apple apps like Photos. PKToolPicker gives you colour and drawing tool pickers, a ruler, a lasso selection tool and an eraser. Additionally, it includes a built in undo/redo functionality, and the toolbox can be moved around the screen and docked to edges by the user. All of this functionality comes for free - no need for you to write any code.

PKCanvasView

PKCanvasView is (as you might expect) a class that represents a canvas onto which the user can draw. It's a UIView subclass (more precisely, a UIScrollView subclass) so can be added to your normal view hierarchy at any arbitrary size, which may be smaller or larger than the actual content size. You'll generally use PKCanvasTool with PKToolPicker and can set up a sophisticated drawing environment with as few lines of code as below:

PKCanvasView takes input from a user directly or via an Apple pencil and maintains a PKDrawing representing the drawn content. Although it's possible to get a bitmap representation of drawn content out of PKDrawing, internally it maintains a vector-like representation. This allows for several nice behaviours, including smart selection/smart modification, and automatic recolouring. For example, in the below video (slightly sped up) the lasso tool locks to the sun's rays and allows them to be dynamically recoloured and moved:

By default, PKCanvasView automatically recolours its content in response to theme changes. For example, the below attempt at a Xamagon recolours based on the selected theme:

If you add a delegate to your PKCanvasView, you can be notified of changes to the drawing as the user makes them. You can get an bitmap representation of of a PKDrawing using the GetImage method. For example, in the below demo (sped up), changes to a drawing are used to create a tiling background, and each subsequent image is layered and animated in a different direction, giving a space-like effect:

In an earlier post I also demonstrated using PKCanvasView with ARKit and the demo app includes a simplified version of that (no repl required :))

Although these demos have mostly been focused on creating new drawings, PKCanvasView can also be used for markup, by giving it a transparent background and placing it over other content.

ARKit3

(WWDC Reference: Introducing ARKit3)
ARKit3 is the third iteration of Apple's augmented reality (AR) framework for mobile devices. In this sense, it is more evolution than revolution, but still includes a large number of welcome improvements across a range of areas, including increased performance and accuracy, enhancements to multiuser AR and a new record/replay capability. Several new AR tasks are now supported, including body tracking, people occlusion, multi-camera tracking and automatic coaching.

Demonstrating AR features can pose practical challenges, so I focussed on a few small demos.

ARCoachingOverlayView

A good AR experience begins with having good tracking data and anchors. ARCoachingOverlayView is a new ARKit feature provided by Apple that allows you to embed an automated, consistent, guide-like overlay into your AR experiences, to help users orient themselves and ARKit correctly.

You make use of the coach by linking an ARCoachingOverlayView to your ARSession and providing it an ARCoachingGoal. Whenever the coach detects that the goal is not met, it will automatically display and guide the user towards the outcome.

ARCoachingOverlayView can be given a delegate, which will call back in response to events such as activation or deactivation of the coach. You can respond to these in order to update your interface (e.g. remove distractions) to help the user focus on tracking.

Since the coach will presumably be used by Apple's own AR application as well as other third parties, I guess the idea is that users will become familiar with this sort of onboarding.

People Occlusion

With the new person segmentation capability, ARKit3 is able to detect people in a frame and their distance from the camera ("depth"), in order to have people and virtual content occlude each other appropriately. The below demo demonstrates this new capability, first with segmentation disabled (to convey the problem) and then with segmentation enabled. Note that segmentation generally performs better for people (and larger parts of people, like bodies) that are further from the camera than what I demonstrate here.

Enabling segmentation is as easy as setting the appropriate flag on your ARWorldTrackingConfiguration:

You can also access estimated depth and segmentation data from ARKit with each frame. In the above video this was simply displayed towards the top of the screen, but if you are clever there might be other things you can do with it.

Multi-camera tracking

In previous versions, ARKit has been limited to using a single camera at a time for AR purposes. Most tasks, such as world tracking, image and object detection, and image tracking, are all performed using the front camera. Face tracking, relies on technologies present in the front camera only. In the past, you would need to use an ARFaceTrackingConfiguration to perform face tracking, meaning that performing sophisticated world tracking at the same time was off the table. In ARKit3, you can now make use of both cameras simulateneously during AR work, making it possible to combine front and back camera tasks. ARWorldTrackingConfiguration has a new property, UserFaceTrackingEnabled which when set causes the front camera to provide face tracking input to the AR session.

You are notified of face anchors in the same way that you are when using an ARFaceTrackingConfiguration, so code you have written for face tracking purposes can still be used here. Combining face tracking with other AR might be useful for allowing the user's expressions or facial movement to influence the scene. Or, you could use it to create creepy floating heads that match your own movement in 3D space. The possibilities are endless.

(Honourable Mention) RealityKit

RealityKit is a new 'single-experience-focused' (my words) framework for AR. Compared to the typical arrangement of ARKit + SceneKit, RealityKit provides a simplified API and a various constraints that make creating AR experiences easier. RealityKit the framework is supported by a new application, 'Reality Composer' which provides a GUI for defining RealityKit projects and can be used on macOS or on iOS devices. The RealityKit APIs do not appear to be bound in current Xamarin.iOS previews, maybe because they are Swift-only. I wonder if there will be an answer for this by release.

CoreML3

(WWDC Reference: Core ML 3 Framework)
Like ARKit, Apple's CoreML framework comes with a bunch of improvements, many of which I cannot claim to understand completely.

• Protocol extensions: CoreML3 brings with it version 4 of the CoreML protocol, which includes support for several new model types and a major bump in the number of neural network layer types, making it a compatible conversion target for more external modes.

• On-device model personalisation: A certain subset of CoreML model types can now be marked as updatable. An updatable model can be deployed with your app, augmented with new examples collected from the user, and retrained in situ, without need for connectivity or external services, and without data leaving the user's device.

• Improvements to CoreML tooling: Both CreateML (Apple's GUI/Wizard-based model training tool), and Turi Create (Apple's python-based model training library) have received several enhancements, in the talk I looked at the former.

Easy Sound Classifier training with CreateML

A new Sound Classifier wizard has been added to CreateML, making it easy to train CoreML models that can categorise audio. To demonstrate this, I used the wizard to train a model that could recognise categories of sound from the Freesound General-Purpose Audio Tagging Challenge. The challenge dataset included approximately 9,000 training examples across 41 categories (applause, finger clicking, keys jangling, barking and many others). With a little preprocessing (generating the folder/file structure that CreateML expects), this dataset dropped straight into CreateML for training without any issues.

Training the model took about two hours on my sacrificial Catalina MBP 2016 and evaluated with reasonable, but not incredible, results. Later I read that about 60% of the training labels in the dataset have not been manually verified. Mislabelled data would influence the quality of the model and evaluation metrics.

To improve the model, I ended up adding an additional 'Ryan' category, trained on audio from my Introduction to ARKit talk at the Melbourne Xamarin Meetup. As we'll see, it did a pretty good job at detecting me.

Live Audio Recognition with SoundAnalysis

SoundAnalysis new framework in iOS13 that simplifies the process of using a CoreML model for sound classification. It takes a trained model (see previous section) and an audio source (either samples, or streaming/recorded audio), and attempts to classify the audio using the model.

Although the documentation from Apple is currently light on, it's fairly straightforward to use a trained model with SoundAnalysis to classify live audio. The feature is a collaboration between SNAudioStreamAnalyzer which performs analyis based on your model, and AVAudioEngine which provides the input (e.g from the microphone). A DidProduceResult callback gives you access to classification data.

The results are an NSArray of SNSoundClassificationResults that are essentially pairs of possible classification and confidence, an example printed to the console below:

{
  "Finger_snapping": 0.77,
  "Scissors": 0.16,
  "Bus": 0.02,
  "Computer_keyboard": 0.02,
  "Bark": 0.01
}

I found the trained model tended produce a lot of false positives (poor precision), but it's worth noting that the dataset was tailored towards sample classification, not neccessary classification of streaming audio. You can see the final model in use below:

warning: contains very low quality audio recorded by laptop mic
also me attempting to bark like a dog
you were warned

This was a 'good' take, and the model did not always perform as well as I'd like. By increasing the threshold for 'positive classification' and potentially smoothing predictions (e.g. only consider a sound classified if receiving multiple sequential confident classifications in a row), it should be possible to reduce the false positive rate.

Demo App

The "Hello iOS13" app containing demos is on Github: rdavisau/hello-ios13.

Slides

Links for the slides are below.

Slides (31): PDF

There's plenty of benefit to trying out the previews, but like every non-Xamarin.iOS 12.7.1.x build they suffer from one drawback - no Reflection.Emit!. You might be thinking - "What good is a preview release if I can't hot reload new PencilKit features in ARKit3 from an embedded REPL?!?" - which is exactly what I thought too. Don't worry though, if we're happy to get our our hands dirty, we can bake our own Xamarin.iOS Xcode 11 Preview with Reflection.Emit available, and even skip ahead of the official preview release cycle while we're at it!

Do I need to bother with this?

Maybe not - particularly if your use case isn't the aforementioned 'hot-reloading new PencilKit features in ARKit3 from an embedded REPL'. If you want to beat the preview release cycle, or 'just' want to use the interpreter, you're good. As Alex Soto mentioned on the last community standup (a great watch on if you want to learn more about how Xamarin.iOS is put together), you can go to the xcode11 branch of xamarin/xamarin-macios, and download the builds from any recent commit. Per the standup, the commits in this branch have been through some internal review and the test suites before merging - so although the builds are not official previews, the quality should be reasonable.

So how to decide? A month or so ago I gave a talk on Practical Uses for the Mono Interpreter, and included a separation of the what needs interpreter and what needs SRE:

Activities on the left side rely on "using the interpreter to 'execute' IL (i.e. non AOT'd code)". This is what you get from --interpreter and is possible in all Xamarin.iOS builds today, including the previously mentioned xcode11 branch builds. Activities on the right side rely on "generating IL, then using the interpreter to 'execute' it", which requires Reflection.Emit is left in the Mono runtime. Alongside the original interpreter announcement, the Xamarin team made sure the Xamarin.iOS 12.7.1.x series of builds included the appropriate bits. IL generation is a mechanism used by hot reload tools like Continuous, and is also useful for enabling REPL-like environments and arbitrary runtime code execution. If you want to play with these kind of things - on the device then yes, you do need to make your own build and should keep reading.

Baking a Xamarin.iOS version with Reflection.Emit (SRE) included

Building your own version of Xamarin.iOS sounds scary, but it's actually pretty straightforward. It's also well documented on the repo wiki, so besides the two changes needed to enabled SRE, I won't be telling you too much more than what you can find there - but I will give you the step by step.

It probably goes without saying, but don't use your custom build of Xamarin.iOS for anything important. You almost certainly don't want to cut release app builds with it - so if you put this on a machine that also acts as a build agent, make sure to take it out of the queue. You can undo the changes we make here by rolling back to a Stable channel build in VS4Mac, but I'm not sure how thorough of a rollback that is (it could be fine, I just don't know) - so continue at your own risk.

Depending on how many of the prerequisites you have, how fast your internet is, and how fast your machine is, I'd expect below steps to take between one and three hours.

Step 1: Have a Mac running Mojave

Catalina drops 32-bit application support and won't run Xcode 9.4. But, Xamarin still relies on/supports a few 32 bit pieces. Xamarin is planning on deprecating all 32-bit support in the true Xamarin.iOS 13.x releases, so this restriction will only be in place during the preview period. The upshot is you need to create your build on Mojave at the moment - don't waste time like I did trying to work around it.

Step 2: Download all the Xcodes

If you're already doing Xamarin.iOS work, you probably at least have Xcode 10 now. We don't need that where we're going, but we do need Xcode 9.4 and Xcode 11 beta X - where X matches the beta release you're targeting. X is is determined by Apple's releases, the Xamarin Mac/iOS team progress and which xamarin-macios commit you build off. At the time of writing, the latest commits in xcode11 are targeting Xcode 11 beta 3. If you want to go back to an earlier commit (e.g. the one that matches the current first preview), you'd need beta 2.

I like to use xcode-install to get different versions of Xcode, but it can't handle the current betas. Rather than dig around the Apple Developer portal for the download links, I recommend using https://xcodereleases.com to quickly get the versions you need.

Once you have both builds, make sure they end up in Applications suffixed with the version. For example:

• Xcode 9.4 - Xcode94.app
• Xcode 11 Beta 3 - Xcode11-beta3.app

Your Applications folder might look a bit like this:

Since we don't need to work with Xcode 10, we don't need to rename it

Step 3: Clone xamarin-macios

In Terminal, navigate to a place you like to put repos (I like ~/Source) and clone xamarin/xamarin-macios:

git clone --recursive https://github.com/xamarin/xamarin-macios

Wait some time while the repo clones, then cd in to the folder, and change to the xcode11 branch, OR checkout the specific commit that you're interested in.

cd xamarin-macios  
git checkout xcode11  

(If you wanted to build a preview that matches the first officially released preview but with SRE, you'd checkout 903c3eec31bc7c9f819f958df5f2a567b2d30631)

Step 4: Set up dependencies

This is pretty much taken straight from the build guide. The xamarin-macios repo has a handy script called system-dependencies.sh which checks for the presence of a build-friendly environment and can automatically resolves some issues, or suggests the resolution. I usually run with the --provision-all flag to get this outcome:

./system-dependencies.sh --provision-all

If it's your first time building for the preview, it's likely that you'll need to run xcode-select to point at your preview Xcode. The system-dependencies.sh script will give you the exact invocation, so you can copy it from there. Again straight from the guide, you also need a few other dependencies which can be installed using HomeBrew. If you don't already have HomeBrew you can use this command to install it (the official HomeBrew page also directs you to install it in this manner):

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"  

Then install the needed bits:

$ brew update
$ brew install libtool autoconf automake bison flex cmake

With those installed, we're ready to start making our changes.

Step 5: Update tools/mtouch/mtouch.cs (optional, for now)

As I mentioned earlier, there are two changes we need to make to get SRE powers. One is in the configuration of our Mono build, which we'll look at shortly. The other is in the argument handling of mtouch.cs, which currently prevents us from passing --enable-repl to device builds and builds with the linker enabled.

These restrictions make sense in an AOT only world, but when we have interpreter powers they get in our way, so we can remove these lines. I say this step is optional because the handling of the --interpreter flag currently automatically adds --enable-repl. However, it was not this way prior to the 12.7.1.x series of builds, and it may change again in the future - strictly you don't need --enable-repl to make use of the interpreter and may prefer to also remove that additional logic from the --interpreter flag. In any case, I've included this step in case it becomes neccessary in the future.

Step 6: Build with SRE in (technically, without SRE out)

At this point, we're more or less ready to go. All we have to do is remove the removal of SRE. There are probably many ways that this can be done; my approach is to take a sledgehammer to external/mono/configure.ac. Among other things, this file determines which defines are set on the mono build, including DISABLE_REFLECTION_EMIT, which we can see is used across various files to bake out the capability.

The heavy handed solution is to just remove the tests that apply the flags - regardless of what is being built - at the lines that look like this:

The challenge is that when we call make world, the external/mono submodule will be reset to a specific commit, so changing it before then will have no effect. There is probably a better way to do it, but since we're only doing it once, my approach is to just update the file after the submodule has been reset. To do that:

• run make world
• wait until you see that external/mono has been checked out (the first time this will take a while)
• press Control+Z to pause the script and return to the prompt
• make the change to external/mono/configure.ac
• run fg to resume execution.

Now we're good to go.

Running make world takes a l o o o n g time, so be patient. I've found that from time to time it fails when running tests, but that is not necessarily fatal, and a subsequent make all -j8 usually succeeds. Once the build succeeds, two commands (again straight from the build guide) will get your new Xamarin.iOS build installed:

make fix-install-permissions # probably only needed the first time  
make install-system  

Seeing something like this tells you it should have worked!

If you want to go back, 'updating' back to a stable channel build using VS4Mac should do the trick. I'm not sure whether it is a completely clean rollback though.

Step 7: Try it out!

If you made it this far, congratulations! You're ready to try your new SRE-powered build. First you should check that everything looks hunky dory in the About window. Note that if you provisioned all as instructed, you may find you now have a Visual Studio For Mac (Preview) app, and should probably use that.
You can tell that your version is installed by checking the build date (should be today if you built it today), and that the hash matches the commit hash you built against. You can now try using the interpreter samples, or techniques I've talked about in previous posts. And of course, you can use them with new iOS13 features, like PencilKit and ARKit3. So I finally do get my REPL Arkit PencitKit extravagana:

Still, discussion regarding code and XAML occurs quite frequently in various arenas. Lately, the presence of alternatives like Flutter and Apple's recent SwiftUI announcement have brought more attention to the topic, as well as increased the discussion around alternate architectures like MVU. I watch these with interest but I find my motivations are simpler - I write my Forms UI in code because before that I was writing iOS UIs in code, and I like to use Continuous for hot reload. When I started introducing Xamarin.Forms into my projects using Forms Embedding, continuing to use code was a straightforward decision.

A twitch stream

David Ortinau spotted the use of coded UI in the Xamarin.Forms UI challenges and invited me to guest on his Twitch stream and demonstrate what the experience of writing your UI in code can be like. You can watch back the stream, in which I walked through building out the login/signup page for Xappy here.

There were two helpers I used in the stream to improve quality of life when writing UI code, @vincenth_net's CSharpForMarkup and @praeclarum's Continuous, which I'll talk a little about here.

CSharpForMarkup

Using C# to build Xamarin.Forms UIs directly can be a little awkward, and one of the projects I quickly came across when first starting out with code was CSharpForMarkup - a set of helpers that aims to let you use a declarative style of C# code for Xamarin Forms UI. CSharpForMarkup wraps and abstracts Xamarin.Forms UI actions that are unweildy to handle in code, allowing you to write concise, readable, declarative and maintainable user interfaces in C#. Based on your preference, you can use the helpers to produce 'code that looks like XAML', or you can produce code that favours conciseness whilst still remaining readable. This example from the readme demonstrates that idea a little:
Being fully contained within a single .cs file of helper methods, CSharpForMarkup is easily integrated into a new or existing project. You can use them 'just' as helper methods, or you can apply the techniques and practices described on the readme page to help drive a consistent code style across your project. One major peril of writing coded UI is the fact that with code 'anything is possible', including overly clever or specialised solutions. With CSharpForMarkup, you can apply and enforce a consistent, coded-based style for your UI development, and produce code that makes you feel good writing it.

If you have only created UIs in #XamarinForms using XAML, you should check out @rdavis_au code up UIs with @vincenth_net’s CSharpForMarkup (https://t.co/woDFSrwZDG). According to Ryan “It’s the kind of thing that will make you feel good and happy about what you are doing.” https://t.co/HSMpnaadit

— Michael Stonis (@MichaelStonis) June 21, 2019

As you might guess, use of CSharpForMarkup was a key focus of the stream. Afterwards, Vincent offered to give the code the full CSharpForMarkup treatment - with all the conventions he recommends - and the final result now lives in Xappy. One thing I also did on the stream is demonstrate a "DSL"-style of UI specification, which relied on dedicated helper methods to reduce the amount of boilerplate in the class. Although I tend to think the declarative style is the right balance of consistency / conciseness / maintainability, in specific cases you might find a DSL style justified.

Continuous

Continuous is a hot reload plugin created by the very excellent @praeclarum several years ago. I have used it on many projects since then and help keep it alive by updating the VS4Mac plugin when the IDE goes through major changes (moving to roslyn, using the new code editor). I am a huge fan of Continuous and use it whenever I work on apps. Besides the stream above, you can see some examples of Continuous' power in videos I've posted in the past:

Over that time, I've learned how to make Continous sing, and have used it in all manner of scenarios, but the methods for doing so are not always pretty. I am used to them all but have to hestitate when I think about the experience that someone new might have after watching my examples. From time to time I have worked on a rethink of Continuous that addresses some of the main pain points (who knows if I will ever finish it), and as an experiment I ported some of those ideas back to the version I used on stream, which is what allowed for the deeper and cleaner integration with Xamarin.Forms Shell. Given the effectiveness of these changes, and how easy they were to integrate, I'd like to apply them back to the core library and maybe do a write up of how to effectively make use of them.

A presentation

Off the back of the Twitch stream, I also gave a quick talk on the same topic at the Queensland C# Mobile Developers meetup group's June 2019 Meetup. This one was a little more hurriedly put together than my normal talks, but (hopefully) gave a good walkthrough of CSharpForMarkup by example and focussed on specific code-based scenarios with a little bit of live coding to demonstrate how the library improves them. Slides for the talk are below.

(Yes, the fringe languages comment was a joke - I love my F#).

Slides (28): PDF

Improved Dev Build Iteration Time

Enabling the interpreter on iOS disables the AOT compilation step for any interpreted assemblies. Since performance is not typically the primary concern at development time (and Debug builds already don't represent Release performance), trading AOT performance for reduced build times is an attractive option when iterating on development of device based features. The timesave when skipping AOT can be substantial, as demonstrated by my completely unscientific and inadmissable measurements that if nothing else are at least indicative.

Practicality: 10/10 highly recommended, few downsides

Hot Reload

The interpreter allows for 'execution' of IL without JIT'ing, allowing us to use the perform code-based hot reload on device-only features using tools like Continuous. In the talk I demonstrated the improved performance of hot reload on the device for frameworks like SpriteKit. I've written more about hot reload and the interpreter in an earlier blogpost

Practicality: 7/10 highly recommended, but seemingly a lost art

Hot Patching

Truly transparent hotpatching is something that would likely require crazy runtime-level acrobatics. However, rolling your own hotpatching implementation is viable with support from the interpreter (whether it is advisable is an entirely different question). A home grown hotpatch implementation relies on abstraction or dedicated patch interception mechanisms. Fortunately, .NET and MVVM development favours abstraction-oriented patterns like DI and VM-based navigation, which are good examples of abstractions that can effectively be patched. In the talk, I demonstrated the remote hotpatching of two apps:

• AR Bound, which was patched to include a new menu screen and two new demos

• A basic Prism app, which was patched with a replacement service layer implementation, and a replacement XF Content Page + ViewModel.

Whilst very effective, hot patching comes with a host of risks and uncertanties. The author struggles find the bravery within to try using this technique outside of a dev/QA environment.

Practicality: -7/10 (yes negative) so enticing, so scary

Embedded REPL

Though the interpreter as exposed through Xamarin.iOS is an IL interpreter, we can approximate a C# REPL by making use of the Mono Evaluator, which compiles and loads IL generated from C# source code. With mixed-mode execution, this IL can then be executed by the interpreter. For this use case I presented a demo of a REPL that could perform various C# evaluations - including UI - but also interact with the current app context.

Although I forgot to present it (😅), the demo included a basic remote execution capability as well, powered by SignalR.

Whilst the practicality of typing code on soft-keyboard is low, the fun factor is very high.

Practicality: Int64.MaxValue/10 REPLs are scientifically proven to be the coolest things in the known universe.

Slides

Links for the slides are below.

Slides (46): PDF

The basics

DumpEditable is essentially a property editor extension that lets you dump an editable representation of an object to the results view, so that you can then modify it interactively and respond to changes in your query. In the spirit of LINQPad's own Dump extension, DumpEditable is available on all objects (with various caveats that I am still exploring), and returns the original object - so can be chained or placed in the middle of a pipeline. The idea is that for many basic cases, what DumpEditable gives you out of the box will be 'good enough', letting you get interactivity 'for free' and allowing you to focus on writing query logic. For example, here's the output you get from dumping a basic POCO with a range of properties including strings, numbers, dates, enums and booleans:
Pretty cool, right? Without writing any special code, we got a fully editable implementation of our Pet, including reasonable handling of enums, nullable booleans and the collection of strings. Because it's handy to work with anonymous types in LINQPad, DumpEditable works with those too, and even allows them to be modified - something you can't do from C# code generally.

If your query has a main loop (like all the gratuituous examples) you can read updated values in the loop body. Otherwise, you'll probably want to be notified when changes are made to your object. You can do this by taking a reference to the EditableDumpContainer that is created by DumpEditable (which is available as an out parameter of DumpEditable) and using one of the three change notification methods it provides to be notified of updates.

Customisation

Whilst DumpEditable aims to play in the 'near enough is good enough' space - giving you cheap interactivity with low to no fuss - I have tried to design it with extensibility in mind. Internally, DumpEditable uses the (poorly named?) concept of EditorRules to determine how to present an editable version of an object. An EditorRule consists of a match function, which returns true if the rule should be used for a given property, and a getEditor function, which returns the content (the 'editor') that should be rendered by LINQPad proper for matching properties. As a user of DumpEditable you can add your own EditorRules, and these will take precedence over any exisiting ones. Writing one from scratch looks something like this: There are a few helpers in the library to make adding your own editors easier, many are covered in the README and demonstrated in the built-in samples, including the slider control:

Cool! How can I use it?

You can install DumpEditable via NuGet: DumpEditable.LINQPad. DumpEditable comes with samples that will automatically be added to your LINQPad samples pane. There's also more documentation at the Github repo. The focus of DumpEditable to date has been centered on my own use caes, so if you think Im missing something that should be handled by DumpEditable, please let me know on GitHub. Happy Editing!

Bonus: Gratuitous uses of LINQPad

Below are a few clips of some wilder LINQPad queries I've put together:

• (left) Internet is down game "AI"
• (right) Automated video game translator
• (bottom) Automated Super Smash Brothers Classic Mode tracker

Visual Challenge

Arranged by David Ortinau - Senior Program Manager on the Forms team and deliverer of many an entertaing presentation - the Visual Challenge was centered on the new Xamarin.Forms 'Visual' feature, which aims to provide a visually identical, or nearly identical, UI experience on iOS and Android out of the box. Visual is realised through 'themes', the first one being focussed on by the team being a Material Design style.

The ask of the Visual Challenge was to use the new feature to replicate an existing screen and provide a comparison of the iOS and Android outputs as well as a write up on the experience. For the challenge, I chose to use a screen from the Qantas Frequent Flyer app, an app I regard to be quite asthetically pleasing. Though I didn't strive for a perfect reproduction (e.g. notice that I didn't recolor some images), the results overall were pretty good:

The screen was easy to reproduce using Xamarin.Forms layouts and controls. For me it highlighted the importance of a good clean design, and how sensible defaults (which Material Visual offers over stock XF) can guide the user towards that. My primary piece of feedback on Visual was that defaults for some controls (like Label.TextColor) were not the same between platforms, which the team is planning to address. The other thing I would have liked is a more consistent tab bar appearance, and/or ability to customise it.

As the only submission that contained a coded UI (yes, I checked every single other submission), I got a little shoutout from David on Twitter:

I know XAML is super popular for creating UI in #XamarinForms. Sometimes it's educational (even inspirational) to see another approach. Take a look at what @rdavis_au did in our VisualChallenge using @vincenth_net's CSharpForMarkup: https://t.co/OG1pRANP6I

— David Ortinau (@davidortinau) March 31, 2019

Look at all the likes! I know you coded UI folks are out there... somewhere..

There were lots of other good submissions to the Visual Challenge, many of which can be seen in the roundup post.

CollectionView Challenge

Hot on the heels of the Visual Challenge, Paul DiPietro (PM on the Forms team) put out a similar challenge focussed on the new CollectionView control. CollectionView is the XF4 successor to ListView, boasting a simpler API with more power and flexibility with respect to layout options.

The CollectionView Challenge was similar in concept to the Visual Challenge - take an existing ListView, or screen with a ListView-like structure, and recreate it using CollectionView - again, with comparison screenshots and a writeup on the experience. Once more, I chose to use a screen from the Qantas app:

Although the layout was not terribly complex, it was a good demonstration of the benefit provided by CollectionView because of the grid-like arrangement - prior to CollectionView this would not be possible with out of the box XF controls. I found the control easy to work with and performance to good. My primary feedback on the CollectionView was desire for a hook or hooks to better allow content to be animated - the lack of which being my primary issue with the existing ListView control and the reason that even in Forms Embedding apps I still develop list-based screens natively. Whilst I was able to add a basic fade-in effect using a bit of a hack, it had some undesirable qualities (content 'fading in again' when scrolling back) and would be difficult to extend beyond this. I typically like to stagger animations or use item position to influence the way it appears, and I think there's an opportunity to allow that capability from XF.

Update: after the submission, I took a crack at hacking staggered entry animations in without modifying CollectionView itself, to prove the concept:

The CollectionView challenge technically runs until the end of April, after which I expect there'll be a roundup post similar to that for Visual Challenge.

Overall

I was impressed by the new Forms 4 features. The combination of Visual, a solid CollectionView class (preferably with hooks to enable the kind of animation I described) are excellent additions. Beyond that, my main want is the ability to do shared element transitions - it appears Javier Suárez Ruiz has been experimenting with exactly this and I eagerly await the write up:

Playing with shared element transitions and animations. In addition to BindableLayout, CollectionView, etc. Soon article and sample. #XamarinForms pic.twitter.com/wo9lxFKJvN

— Javier Suárez Ruiz (@jsuarezruiz) April 19, 2019

With all this in the box, the needle continues to swing on choice of Xamarin.Forms vs Xamarin.Native. I am a big fan of my 'Native app shell, Forms-embedded pages' approach, because of the escape hatches it provides. However, as XF Shell matures, and the benefits of letting that be someone else's problem increase, abandoning even embedding becomes a possibility. In any case, I'm pleased to see the progress and to have good options available.

This time the meetup was streamed on Twitch, potentially signalling my transition from chatroom memer to full-time streamer (but probably not). It is also now published on YouTube, so you can watch it back at your leisure (~1h):

Again, I added a new demo to the the Earthbound-themed demo app to keep it fresh. This time we looked at 3D object detection. I found it a bit flakey and it didn't go perfectly during the talk, but with a bit of luck the next iteration of ARKit will improve it. The updated source is on Github, and a demo of this feature is here:

(Since there is no Ninten amiibo, I used the Xamarin monkey as a stand in - it was actually a lot better for object detection; amiibo may be a little too small).

Links for the slides are below.

Slides (42): PDF