-
Dos and Don'ts of stackalloc
In .NET Core 2.1 a small but well-received feature was the ability to “safely” allocate a segment of data on the stack, using
stackalloc
, when used withSpan<T>
.Before
Span<T>
,stackalloc
required being in anunsafe
context:unsafe { byte* data = stackalloc byte[256]; }
The use of
unsafe
, along with the little number of APIs in .NET that could work with pointers, was enough to deter a lot of people from using it. As a result, it remained a relatively niche feature. The introduction ofSpan<T>
now means this can be done without being in anunsafe
context now:Span<byte> data = stackalloc byte[256];
The .NET team has also been working diligently to add
Span<T>
APIs where it makes sense. There is now more appeal and possibility to usestackalloc
.stackalloc
is desirable in some performance sensitive areas. It can be used in places where small arrays were used, with the advantage that it does not allocate on the heap - and thus does not apply pressure to the garbage collector.stackalloc
is not a general purpose drop-in for arrays. They are limited in a number of ways that other posts explain well enough, and require the use ofSpan<T>
.Recently I vented a bit about
stackalloc
on Twitter, as one does on Twitter, specifically the community’s fast embrace of it, without discussing or well-documenting some ofstackalloc
’s sharp edges. I’m going to expand on that here, and make an argument forstackalloc
still being unsafe and requiring some thought about being used.DON’T: Use variable allocation lengths
A large risk with using
stackalloc
is running out of stack space. If you’ve ever written a method that is recursive and went too deep, you’ll eventually receive aStackOverflowException
. TheStackOverflowException
is a bit special in that it is one of the exceptions that cannot be caught. When a stack overflow occurs, the process immediately exits. Allocating too much withstackalloc
has the same effect - it causes a stack overflow and the process immediately terminates.This is particularly worrisome when the allocation’s length is determined by user input:
Span<char> buffer = stackalloc char[userInput.Length]; //DON'T
This allows users to take down your process, an effective denial-of-service.
Using a constant also reduces the risk of arithmetic or overflow mistakes. Currently, .NET Core’s CLR interprets that amount to be allocated as an unsigned integer. This means that an arithmetic over or under flow may result in a stack overflow.
Span<byte> b = stackalloc byte[(userInput % 64) - 1]; //DON'T
DO: Use a constant for allocation size
Instead, it’s better to use a constant value for
stackalloc
, always. It immediately resolves any ambiguities about how much is allocated on the stack.Span<char> buffer = stackalloc char[256]; //better
Once you have an allocated buffer, you can use Span’s
Slice
funtionality to adjust it to the correct size:Span<char> buffer = stackalloc char[256]; Span<char> input = buffer.Slice(0, userInput.Length);
DON’T: Use stackalloc in non-constant loops
Even if you allocate a fixed length amount of data on the stack, doing so in a loop can be dangerous as well, especially if the number of the iterations the loop makes is driven by user input:
for (int i = 0; i < userInput; i++) { // DON'T Span<char> buffer = stackalloc char[256]; }
This also can cause a denial of service, since this allows someone to control the number of stack allocations, though not the length of the allocation.
DO: Allocate outside of loops
Span<char> buffer = stackalloc char[256]; //better for (int i = 0; i < userInput; i++) { //Do something with buffer }
Allocating outside of the loop is the best solution. This is not only safer, but also better for performance.
DON’T: Allocate a lot on the stack
It’s tempting to allocate as much as nearly possible on the stack:
Span<byte> data = stackalloc byte[8000 * 1024]; // DON'T
You may find that this runs fine on Linux, but fails on Windows with a stack overflow. Different operating systems, architectures, and environments, have different stacks limits. Linux typically allows for a larger stack than Windows by default, and other hosting scenarios such as in an IIS worker process come with even lower limits. An embedded environment may have a stack of only a few kilobytes.
DO: Conservatively use the stack
The stack should be used for small allocations only. How much depends on the size of each element being allocated. It’s also desirable to not allocate many large structs, either.
I won’t prescribe anything specific, but anything larger than a kilobyte is a point of concern. You can allocate on the heap depending on how much you need. A typical pattern might be:
const int MaxStackSize = 256; Span<byte> buffer = userInput > MaxStackSize ? new byte[userInput] : stackalloc byte[MaxStackSize]; Span<byte> data = buffer.Slice(0, userInput);
This will allocate on the stack for small amounts, still in a constant amount, or if too large, will use a heap-allocated array. This pattern may also make it easier to use
ArrayPool
, if you choose, which also does not guarantee that the returned array is exactly the requested size:const int MaxStackSize = 256; byte[]? rentedFromPool = null; Span<byte> buffer = userInput > MaxStackSize ? (rentedFromPool = ArrayPool<byte>.Shared.Rent(userInput)) : stackalloc byte[MaxStackSize]; // Use data Span<byte> data = buffer.Slice(0, userInput); // Return from pool, if we rented if (rentedFromPool is object) { // DO: if using ArrayPool, think carefully about clearing // or not clearing the array. ArrayPool<byte>.Shared.Return(rentedFromPool, clearArray: true); }
DON’T: Assume stack allocations are zero initialized
Most normal uses of
stackalloc
result in zero-initialized data. This behavior is however not guaranteed, and can change depending if the application is built for Debug or Release, and other contents of the method. Therefore, don’t assume that any of the elements in astackalloc
edSpan<T>
are initialized to something by default. For example:Span<byte> buffer = stackalloc byte[sizeof(int)]; byte lo = 1; byte hi = 1; buffer[0] = lo; buffer[1] = hi; // DONT: depend on elements at 2 and 3 being zero-initialized int result = BinaryPrimitives.ReadInt32LittleEndian(buffer);
In this case, we might expect the result to be 257, every time. However if the
stackalloc
does not zero initialize the buffer, then the contents of the upper-half of the integer will not be as expected.This behavior will not always be observed. In Debug builds, it’s likely that you will see that
stackalloc
zero-initializes its contents every time, whereas in Release builds, you may find that the contents of astackalloc
are uninitialized.Starting in .NET 5, developers can opt to explicitly skip zero-initializing
stackalloc
contents with theSkipLocalsInit
attribute. Without it, whether or notstackalloc
is default initialized is up to Roslyn.DO: Initialize if required
Any item read from a
stackalloc
ed buffer should be explicitly assigned, or useClear
to explicitly clear the entireSpan<T>
and initialize it to defaults.Span<byte> buffer = stackalloc byte[sizeof(int)]; buffer.Clear(); //explicit zero initialize byte lo = 1; byte hi = 1; buffer[0] = lo; buffer[1] = hi; int result = BinaryPrimitives.ReadInt32LittleEndian(buffer);
Though not explicitly covered in this post, the same advice applies to arrays rented from the
ArrayPool
.Summary
In summary,
stackalloc
needs to be used with care. Failing to do so can result in process termination, which is a denial-of-service: your program or web server aren’t running any more. -
Import and Export RSA Key Formats in .NET Core 3
.NET Core 3.0 introduced over a dozen new APIs for importing and exporting RSA keys in different formats. Many of them are a variant of another with a slightly different API, but they are extremely useful for working with private and public keys from other systems that work with encoding keys.
RSA keys can be encoded in a variety of different ways, depending on if the key is public or private or protected with a password. Different programs will import or export RSA keys in a different format, etc.
Often times RSA keys can be described as “PEM” encoded, but that is already ambiguous as to how the key is actually encoded. PEM takes the form of:
-----BEGIN LABEL----- content -----END LABEL-----
The content between the labels is base64 encoded. The one that is probably the most often seen is BEGIN RSA PRIVATE KEY, which is frequently used in web servers like nginx, apache, etc:
-----BEGIN RSA PRIVATE KEY----- MII... -----END RSA PRIVATE KEY-----
The base64-encoded text is an RSAPrivateKey from the PKCS#1 spec, which is just an ASN.1 SEQUENCE of integers that make up the RSA key. The corresponding .NET Core 3 API for this is
ImportRSAPrivateKey
, or one of its overloads. If your key is “PEM” encoded, you need to find the base64 text between the label BEGIN and END headers, base64 decode it, and pass toImportRSAPrivateKey
. There is currently an API proposal to make reading PEM files easier. If your private key is DER encoded, then that just means you can read the content directly as bytes in toImportRSAPrivateKey
.Here is an example:
var privateKey = "MII..."; //Get just the base64 content. var privateKeyBytes = Convert.FromBase64String(privateKey); using var rsa = RSA.Create(); rsa.ImportRSAPrivateKey(privateKeyBytes, out _);
When using openssl, the
openssl rsa
commands typically output RSAPrivateKey PKCS#1 private keys, for exampleopenssl genrsa
.A different format for a private key is PKCS#8. Unlike the RSAPrivateKey from PKCS#1, a PKCS#8 encoded key can represent other kinds of keys than RSA. As such, the PEM label for a PKCS#8 key is “BEGIN PRIVATE KEY” (note the lack of “RSA” there). The key itself contains an AlgorithmIdentifer of what kind of key it is.
PKCS#8 keys can also be encrypted protected, too. In that case, the PEM label will be “BEGIN ENCRYPTED PRIVATE KEY”.
.NET Core 3 has APIs for both of these. Unencrypted PKCS#8 keys can be imported with
ImportPkcs8PrivateKey
, and encrypted PKCS#8 keys can be imported withImportEncryptedPkcs8PrivateKey
. Their usage is similar toImportRSAPrivateKey
.Public keys have similar behavior. A PEM encoded key that has the label “BEGIN RSA PUBLIC KEY” should use
ImportRSAPublicKey
. Also like private keys, the public key has a format that self-describes the algorithm of the key called a Subject Public Key Info (SPKI) which is used heavily in X509 and many other standards. The PEM header for this is “BEGIN PUBLIC KEY”, andImportSubjectPublicKeyInfo
is the correct way to import these.All of these APIs have export versions of themselves as well, so if you are trying to export a key from .NET Core 3 to a particular format, you’ll need to use the correct export API.
To summarize each PEM label and API pairing:
- “BEGIN RSA PRIVATE KEY” =>
RSA.ImportRSAPrivateKey
- “BEGIN PRIVATE KEY” =>
RSA.ImportPkcs8PrivateKey
- “BEGIN ENCRYPTED PRIVATE KEY” =>
RSA.ImportEncryptedPkcs8PrivateKey
- “BEGIN RSA PUBLIC KEY” =>
RSA.ImportRSAPublicKey
- “BEGIN PUBLIC KEY” =>
RSA.ImportSubjectPublicKeyInfo
One gotcha with openssl is to pay attention to the output of the key format. A common enough task from openssl is “Given this PEM-encoded RSA private key, give me a PEM encoded public-key” and is often enough done like this:
openssl rsa -in key.pem -pubout
Even if key.pem is a PKCS#1 RSAPrivateKey (“BEGIN RSA PRIVATE KEY”), the
-pubout
option will output a SPKI (“BEGIN PUBLIC KEY”), not an RSAPublicKey (“BEGIN RSA PUBLIC KEY”). For that, you would need to use-RSAPublicKey_out
instead of-pubout
. The opensslpkey
commands will also typically give you PKCS#8 or SPKI formatted keys. - “BEGIN RSA PRIVATE KEY” =>
-
Sometimes valid RSA signatures in .NET
One of the nice things about .NET Core being open source is following along with some of the issues that people report. I tend to keep an eye on System.Security tagged issues, since those tend to be at the intersection of things that interest me and things I can maybe help with.
A user filed an issue where .NET Framework considered a CMS valid, and .NET Core did not. This didn’t entirely surprise me. In the .NET Framework, the
SignedCms
class is heavily backed by Windows’ handling of CMS/PKCS#7. In .NET Core, the implementation is managed (sans the cryptography). The managed implementation adheres somewhat strictly to the CMS specification. As other issues have noticed, Windows’, thus .NET Framework’s, implementation was a little more relaxed in some ways.This turned out not to be one of those cases. The CMS part was actually working just fine. What was failing was RSA itself. The core of the issue was that different implementations of RSA disagreed on the RSA signature’s validity.
That seems pretty strange!
When I talk about different implementations on Windows, I am usually referring to CAPI vs CNG, or
RSACryptoServiceProvider
andRSACng
, respectively. For now, I’m keeping this post to the .NET Framework. We’ll bring .NET Core in to the discussion later.There are two implementations because, well, Windows has two of them. CNG, or “Cryptography API: Next Generation” is the newer of the two and is intended to be future of cryptographic primitives on Windows. It shipped in Windows Vista, and offers functionality that CAPI cannot do. An example of that is PSS RSA signatures.
.NET Framework exposes these implementations as
RSACryptoServiceProvider
andRSACng
. They should be interchangable, and CNG implementations should be used going forward. However, there is one corner case where the old, CAPI implementation considers a signature valid while the CNG one does not.The issue can be demonstrated like so:
byte[] n = new byte[] { ... }; byte[] e = new byte[] { ... }; byte[] signature = new byte[] { ... }; var digest = new byte[] { 0x68, 0xB4, 0xF9, 0x26, 0x34, 0x31, 0x25, 0xDD, 0x26, 0x50, 0x13, 0x68, 0xC1, 0x99, 0x26, 0x71, 0x19, 0xA2, 0xDE, 0x81, }; using (var rsa = new RSACng()) { rsa.ImportParameters(new RSAParameters { Modulus = n, Exponent = e }); var valid = rsa.VerifyHash(digest, signature, HashAlgorithmName.SHA1, RSASignaturePadding.Pkcs1); Console.WriteLine(valid); } using (var rsa = new RSACryptoServiceProvider()) { rsa.ImportParameters(new RSAParameters { Modulus = n, Exponent = e }); var valid = rsa.VerifyHash(digest, signature, HashAlgorithmName.SHA1, RSASignaturePadding.Pkcs1); Console.WriteLine(valid); }
When used with one of the curious signatures that exhibits this behavior, such as the one in the GitHub link, the first result will be false, and the second will be true.
Nothing jumped out at me as being problematic. The signature padding is PKCS, the public exponent is the very typical 67,537, and the RSA key is sensible in size.
To make it stranger, this signature came off the timestamp of Firefox’s own signed installer. So why are the results different?
Jeremy Barton from Microsoft on .NET Core made the observation that the padding in the RSA signature itself is incorrect, but in a way that CAPI tollerates and CNG does not, at least by default. Let’s look at the raw signature. To do that, we need the public key and signature on disk, and we can poke at them with OpenSSL.
Using the command:
openssl rsautl -verify -in sig.bin -inkey key.der \ -pubin -hexdump -raw -keyform der
We get the following output:
0000 - 00 01 ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0010 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0020 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0030 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0040 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0050 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0060 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0070 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0080 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0090 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 00a0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 00b0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 00c0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 00d0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 00e0 - ff ff ff ff ff ff ff ff-ff ff ff 00 68 b4 f9 26 00f0 - 34 31 25 dd 26 50 13 68-c1 99 26 71 19 a2 de 81
This is a PKCS#1 v1.5 padded signature, as indicated by by starting with 00 01. The digest at the end can be seen,
68 b4 f9 26 ... 19 a2 de 81
which matches the digest above, so we know that the signature is for the right digest.What is not correct in this signature is how the digest is encoded. The signature contains the bare digest. It should be encoded as an ASN.1 sequence along with the AlgorithmIdentifer of the digest:
DigestInfo ::= SEQUENCE { digestAlgorithm AlgorithmIdentifier, digest OCTET STRING }
This goes back all the way to a document (warning: link is to an ftp:// site) written in 1993 by RSA labratories explaining how PKCS#1 v1.5 works,and was standardized in to an RFC in 1998.
The RSA signature we have only contains the raw digest. It is not part of a
DigestInfo
. If the digest were properly encoded, it would look something like this:0000 - 00 01 ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0010 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0020 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0030 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0040 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0050 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0060 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0070 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0080 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 0090 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 00a0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 00b0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 00c0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff 00d0 - ff ff ff ff ff ff ff ff-ff ff ff ff 00 30 21 30 00e0 - 09 06 05 2b 0e 03 02 1a-05 00 04 14 68 b4 f9 26 00f0 - 34 31 25 dd 26 50 13 68-c1 99 26 71 19 a2 de 81
The signature now includes
DigestInfo
along with the OID 1.3.14.3.2.26 to indicate that the digest is SHA1.At this point we know what the difference is, and the original specification in part 10.1.2 makes it fairly clear that the “data” should be a digest and should be encoded as DigestInfo, not a bare digest.
The source of this signature is from Verisign's timestamp authority at http://timestamp.verisign.com/scripts/timstamp.dll. After checking with someone at DigiCert (now running this TSA), it was launched in May 1995.
I suspect that the TSA is old enough that the implementation was made before the specification was complete or simply got the specification wrong and no one noticed. Bringing this back to CNG and CAPI, CNG can validate this signatures, but you must explicitly tell CNG that the signature does not have an object identifier.
BCRYPT_PKCS1_PADDING_INFO
’s documentation has the detail there, but gist of it isIf there is no OID in the signature, then verification fails unless this member is NULL.
This would be used with
{B,N}CryptVerifySignature
. To bring this back around to the .NET Framework, how do we useRSACng
and givenull
in for the padding algorithm? The short answer is: you cannot. If you try, you will get an explicitArgumentException
saying that the hash algorithm name cannot be null.For .NET Framework, this solution “keep using
RSACryptoServiceProvider
”. If you need to validate these signatures, chances are you do not need to use CNG’s newer capabilities like PSS since these malformed signatures appear to be coming from old systems. Higher level things likeSignedCms
andSignedXml
useRSACryptoServiceProvider
by default, so they will continue to work.To bring in .NET Core, the situation is a little more difficult. If you are using
SignedCms
like so:var signedCms = new SignedCms(); signedCms.Decode(File.ReadAllBytes("cms-with-sig.bin")); signedCms.CheckSignature(true);
This will start throwing when you migrate to .NET Core. .NET Core will use CNG when run on Windows to validate RSA signatures for
SignedCms
andSignedXml
. This is currently not configurable, either. When used withSignedCms
, it ultimately calls theX509Certificate2.GetRSAPublicKey()
extension method, and that will always return an implementation based on CNG.If you are using
SignedCms
on .NET Core and need to validate a CMS that is signed with these problematic signatures, you are currently out of luck using in-the-box components. As far as other platforms go, both macOS and Linux environments for .NET Core will agree with CNG - that the signature is invalid.The good news is, these signatures are not easy to come by. So far, only the old Verisign timestamp authority is known to have produced signatures like this.
-
C# ReadOnlySpan
and static data Since C# 7 there have been a lot of point releases that contain all kinds of goodies. Many of them are performance focused, such as safe stack allocations using
Span<T>
, or interoperability with improvements tofixed
.One that I love, but is not documented well, is some special treatment that
ReadOnlySpan<byte>
gets when its contents are known at compile time.Here’s an example of a lookup table I used to aide with hex encoding that uses a
byte[]
:private static byte[] LookupTable => new byte[] { (byte)'0', (byte)'1', (byte)'2', (byte)'3', (byte)'4', (byte)'5', (byte)'6', (byte)'7', (byte)'8', (byte)'9', (byte)'A', (byte)'B', (byte)'C', (byte)'D', (byte)'E', (byte)'F', };
This binary data has to get stored somewhere in our produced library. If we use
dumpbin
we can see it in the .text section of the binary.dumpbin /RAWDATA /SECTION:.text mylib.dll
Right at the bottom, we see:
00402A40: 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46 0123456789ABCDEF
I won’t go into the a lot of the details on how this data is compiled into the
.text
section, but at this point we need to get that data into the array somehow.If we look at the jit assembly of
LookupTable
, we see:sub rsp, 0x28 vzeroupper mov rcx, 0x7ffc4638746a mov edx, 0x10 call 0x7ffc49b52630 mov rdx, 0x1b51450099c lea rcx, [rax+0x10] vmovdqu xmm0, [rdx] vmovdqu [rcx], xmm0 add rsp, 0x28 ret
Where
0x7ffc49b52630
isInitializeArray
.With an array, our property leans on
InitializeArray
, the source of which is in the CoreCLR. For little-endian platforms, it boils down to amemcpy
from a runtime field handle.Indeed, with a debugger we finally see:
00007ffd`b18b701a e831a40e00 call coreclr!memcpy (00007ffd`b19a1450)
Dumping
@rdx L10
yields:000001f0`4c552a90 30 31 32 33 34 35 36 37-38 39 41 42 43 44 45 46 0123456789ABCDEF
So that was a very long-winded way of saying that when using arrays, initializing a field or variable with bytes results in
memcpy
from the image into the array, which results in more data on the heap.Now, starting in 7.3, we can avoid that
memcpy
when usingReadOnlySpan<byte>
.private static ReadOnlySpan<byte> LookupTable => new byte[] { (byte)'0', (byte)'1', (byte)'2', (byte)'3', (byte)'4', (byte)'5', (byte)'6', (byte)'7', (byte)'8', (byte)'9', (byte)'A', (byte)'B', (byte)'C', (byte)'D', (byte)'E', (byte)'F', };
Looking at the jit assembly:
mov eax, 0x10 xor edx, edx mov r8, 0x1b5144c0968 mov [rcx], rdx mov [rcx+0x8], r8 mov [rcx+0x10], eax mov rax, rcx ret
We see that there is
mov r8, 0x1b5144c0968
. The contents of0x1b5144c0968
are:000001b5`144c0968 30 31 32 33 34 35 36 37-38 39 41 42 43 44 45 46 0123456789ABCDEF
So we see that the method is now returning the data directly and omitting the
memcpy
entirely, so ourReadOnlySpan<byte>
is pointing directly to the.text
section.This works for property getters as shown above, but also as the return of a method:
ReadOnlySpan<byte> GetBytes() { return new byte[] { ... }; }
Which works similar to the getter of the property. In addition, this also works for locals in a method body as well:
void Write200Ok(Stream s) { ReadOnlySpan<byte> data = new byte[] { (byte)'H', (byte)'T', (byte)'T', (byte)'P', (byte)'/', (byte)'1', (byte)'.', (byte)'1', (byte)' ', (byte)'2', (byte)'0', (byte)'0', (byte)' ', (byte)'O', (byte)'K' }; s.Write(data); }
Which also produces a reasonable JIT disassembly:
sub rsp, 0x38 xor eax, eax mov qword ptr [rsp+0x28], rax mov qword ptr [rsp+0x30], rax mov rcx, 0x1e595b42ade mov eax, 0x0F lea r8, [rsp+0x28] mov qword ptr [r8], rcx mov dword ptr [r8+8], eax mov rcx, rdx lea rdx, [rsp+0x28] cmp dword ptr [rcx], ecx call 0x7ff89ede10c8 (Stream.Write(System.ReadOnlySpan`1<Byte>), mdToken: 0000000006000001) add rsp, 0x38 ret
Here we see
mov rcx, 0x1e595b42ade
which moves the address of the static data directly in to the register with no additional work to create a byte array.These optimizations currently only works with
ReadOnlySpan<byte>
right now. Other types will continue to useInitializeArray
due to needing to handle different platforms and how they handle endianness. -
C# 8 using declarations
Visual Studio 2019 preview 2 was released a few days ago and I took the time to install it. Visual Studio itself is actually rather uninteresting to me, however the inclusion of the next C# 8 preview got my attention. I glanced at the feature highlights and posted “looks nice” on Twitter.
Predictably, I got a few responses like “I’m not sure I like that”, and there is always a guarantee that if F# has a similar feature, an F# developer will appear and tell you F# has had this feature for 600 years.
The one I like a lot is using declarations. This allows a local to automatically be disposed at the end of the block. Essentially, it hides the
try
/finally
or theusing() {...}
. The .NET team’s blog kind of gave a bad example of this, so I’ll use one from Open OPC SignTool. Here is the original snippet:private static X509Certificate2 GetCertificateFromCertificateStore(string sha1) { using (var store = new X509Store(StoreName.My, StoreLocation.LocalMachine)) { store.Open(OpenFlags.OpenExistingOnly | OpenFlags.ReadOnly); var certificates = store.Certificates.Find(X509FindType.FindByThumbprint, sha1, false); return certificates.Count > 0 ? certificates[0] : null; } }
A
using var
can make this:private static X509Certificate2 GetCertificateFromCertificateStore(string sha1) { using var store = new X509Store(StoreName.My, StoreLocation.LocalMachine); store.Open(OpenFlags.OpenExistingOnly | OpenFlags.ReadOnly); var certificates = store.Certificates.Find(X509FindType.FindByThumbprint, sha1, false); return certificates.Count > 0 ? certificates[0] : null; }
This has the same effect of
store
havingDispose
called on it at the end of the method. The benefit here being that there is less indentation and braces. This keeps me focused on the code that matters. I don’t care whenstore
is disposed in the method, I can just observe that it has ausing
modifier on the local and be assured thatDispose
will be called.This isn’t the same as garbage collection or finalizers. Both of those are non- deterministic, and can lead to unexpected program behavior. That’s less so in the case of
X509Store
, so let’s look at another example:using Stream stream = entry.Open(); var xmlDocument = XDocument.Load(stream, LoadOptions.PreserveWhitespace); return new OpcRelationships(location, xmlDocument, readOnlyMode);
Not disposing a stream that is backed by a file can cause access errors later in software that might try to open that file again - it is already open, so not only is it a bad idea it leave streams to the GC, it is just simply incorrect.
However again
using
on the local ensures it is deterministically closed.When it gets disposed I can see being slightly unclear to the developer. The quick explanation is when the local is no longer reachable, not when it is last used. The C# 8 above gets compiled roughly to:
var stream = entry.Open(); try { var xmlDocument = XDocument.Load(stream, LoadOptions.PreserveWhitespace); return new OpcRelationships(location, xmlDocument, readOnlyMode); } finally { if (stream != null) { ((IDisposable)stream).Dispose(); } }
The disposal is done after the return, when the local is no longer reachable, not after
XDocument
is created.I find this very helpful to keep code readable. This doesn’t work when you need fine control over when
Dispose
is called. A place where this does not work well is when theDispose
pattern is used for scopes, such as logging. The AzureSignTool project has code similar to this inSignCommand
:var logger = loggerFactory.CreateLogger<SignCommand>(); Parallel.ForEach(AllFiles, options, () => (succeeded: 0, failed: 0), (filePath, pls, state) => { using (var loopScope = logger.BeginScope("File: {Id}", filePath)) { logger.LogInformation("Signing file."); //Sign the file. omit a bunch of other code. logger.LogInformation("Done signing the file."); } logger.LogDebug("Incrementing success count."); return (state.succeeded + 1, state.failed); }
Here, we cannot change this to a
using var
because then theLogDebug
would be inside of that logging scope, which it wasn’t before. This is a place where we continue to wantDispose
to be called at a different time from the whenloopScope
would no longer be in scope.My impression from C# developers is that they do not tend to call
Dispose
on resources as soon as it can be disposed, just at a reasonable point in the same method. Most developers do not write this code:public bool MightBeExe(string filePath) { var firstBytes = new byte[2]; int bytesRead; using (var file = File.Open(filePath, FileMode.Open)) { bytesRead = file.Read(firstBytes, 0, 2); } return bytesRead == 2 && firstBytes[0] == (byte)'M' && firstBytes[1] == (byte)'Z'; }
They will instead write something like:
public bool MightBeExe(string filePath) { using (var file = File.Open(filePath, FileMode.Open)) { var firstBytes = new byte[2]; var bytesRead = file.Read(firstBytes, 0, 2); return bytesRead == 2 && firstBytes[0] == (byte)'M' && firstBytes[1] == (byte)'Z'; } }
Which is a perfect candidate for
using var
:public bool MightBeExe(string filePath) { using var file = File.Open(filePath, FileMode.Open); var firstBytes = new byte[2]; var bytesRead = file.Read(firstBytes, 0, 2); return bytesRead == 2 && firstBytes[0] == (byte)'M' && firstBytes[1] == (byte)'Z'; }
There are of course some reasonable limitations to this feature. For example, it cannot be combined with out-variables.
if (Crypto32.CryptEncodeObjectEx( // other stuff out var handle, ref size) ) { using (handle) { // Do stuff } }
This does not work:
if (Crypto32.CryptEncodeObjectEx( // other stuff out using var handle, ref size) ) { // Do stuff }
Jared Parsons said on Twitter that C# folks thought of this, and decided that it had “Too much confusion about ownership.” Thinking about it myself, I agree, so it’s nice that the feature is limited in that regard.
Another limitation is that the variable cannot be reassigned. For example:
using var stream = entry.Open(); stream = entry2.Open();
This will produce error CS1656, “Cannot assign to ‘stream’ because it is a ‘using variable’”.
All in all, I very much like this small feature in C# 8. It has reasonable guard rails on it from doing something too weird like re-assigning to it, while giving the benefit of less blocks, braces, indentation.