Using nuget packages to help facade the complexities of some resource dependency can no doubt speed up development, but often it can create a bit of a headache when it comes to unit testing. Anything that uses a lot of static classes, extension methods or a fluent api approach can be somewhat frustrating to mock up to allow you to test the bit of code you have written.
It’s often a topic of contention – unit test coverage ideals versus added value but I think it’s always worth investigating the cost first before slapping a [ExcludeFromCodeCoverage] banner on top of your class. Even if you decide it’s not worth progressing, it might give you a much better understanding of what’s happening under the bonnet of the package upon which you’ve opted to rely.
Whenever I’m looking for nuget packages, I like to see those that have been generated from a public, open source repo, such as Github, partly because it indicates a better chance of lasting support but mainly because I can dip into the code to determine the underlying calls when I need to (such as what to Mock when your code is calling an extension method).
The MongoDb.Driver packages for C# (including Bson and Core) are covered by their open source github repo here, so using it to create my own package meant I was able to dive into the code and determine how (or if!) I could ensure as much of my code was covered by tests as possible. The short of it is that, due to a number of extension methods used by the DbDriver code, testing my base queries proved a little more involved that I first expected! I therefore decided, having come out the other end of the tunnel, it might be worth noting down how I muddled my way through it all.
All of the code for my Mongo Db Persistence nuget package can be found on GitHub and the nu get package can found here.
Filters
One of the aspects I found was that, whether you opt to use the MongoDb FilterDefinitions to build your filters, or opt for the more vanilla linq expressions approach, the resulting json filter is the same. So, where some generic collection of type TDocument exists, the following Find operation against an id result in the same json filter
collection.Find(Builders<TDocument>.Filter.Eq("_id", id)) // is the same as collection.Find(doc => doc.Id == id) // and both create the following json filter var expectedJsonFilter = "{ \"_id\" : CSUUID(\"" + id + "\") }";
Setup
In my project, I’ve facaded as much of the specific as possible to allow for decoupled use and easier unit tests. This has left me with the following interfaces and base classes:
public interface IDbContext : IDisposable { void AddCommand(Func<Task> func); Task<int> SaveChangesAsync(); } public abstract class IndexedEntityBase { public Guid Id { get; set; } } public interface IMongoContext : { IMongoCollection<TDocument> GetCollection<TDocument>(string name) where TDocument : IndexedEntityBase; } public abstract class RepositoryBase<TDocument, TContext> : IRepository<TDocument> where TContext : IDbContext where TDocument : class { private readonly ILogger logger; private readonly TContext context; protected RepositoryBase( TContext context, ILogger logger) { this.context = context; this.logger = logger; } public IDbContext DbContext => context; public abstract void Add(TDocument document); public abstract Task<TDocument> GetByIdAsync(Guid id); public abstract TDocument GetById(Guid id); public abstract Task<IEnumerable<TDocument>> GetAllAsync(); public abstract IEnumerable<TDocument> GetAll(); public abstract void Update(TDocument document); public abstract void Remove(Guid id); public void Dispose() { Dispose(true); GC.SuppressFinalize(this); } protected virtual void Dispose(bool disposing) { if (disposing) { context?.Dispose(); } } }
My Mongo specific repository base class then looks like this:
public class DocumentRepositoryBase<TDocument> : RepositoryBase<TDocument, IMongoContext> where TDocument : IndexedEntityBase { private readonly IMongoContext context; private readonly ILogger logger; protected DocumentRepositoryBase(IMongoContext context, ILogger logger) : base(context, logger) { this.context = context; this.logger = logger; } public override void Add(TDocument document) { ExecuteDbSetAction((ctx, collection) => ctx.AddCommand(() => collection.InsertOneAsync(document))); } public override async Task<TDocument> GetByIdAsync(Guid id) { var result = await ExecuteDbSetFuncAsync(collection => collection.FindAsync( Builders<TDocument>.Filter.Eq("_id", id))) .ConfigureAwait(false); return result.SingleOrDefault(); } public override TDocument GetById(Guid id) { var result = ExecuteDbSetFunc(collection => collection.Find(doc => doc.Id == id)); return result.SingleOrDefault(); } public override async Task<IEnumerable<TDocument>> GetAllAsync() { var result = await ExecuteDbSetFuncAsync(collection => collection .FindAsync(Builders<TDocument>.Filter.Empty)) .ConfigureAwait(false); return result.ToList(); } public override IEnumerable<TDocument> GetAll() { var result = ExecuteDbSetFunc(collection => collection.Find(Builders<TDocument>.Filter.Empty)); return result.ToList(); } public override void Update(TDocument document) { ExecuteDbSetAction((ctx, collection) => ctx.AddCommand(() => collection.ReplaceOneAsync(Builders<TDocument>.Filter.Eq("_id", document.Id), document))); } public override void Remove(Guid id) { ExecuteDbSetAction((ctx, collection) => ctx.AddCommand(() => collection.DeleteOneAsync(Builders<TDocument>.Filter.Eq("_id", id)))); } private void ExecuteDbSetAction(Action<IMongoContext, IMongoCollection<TDocument>> action) { var dbSet = GetDbSet(); // we know the app will throw an exception if the previous statement fails to deliver action.Invoke(context, dbSet!); } private async Task<TResult> ExecuteDbSetFuncAsync<TResult>(Func<IMongoCollection<TDocument>, Task<TResult>> func) { var dbSet = GetDbSet(); // we know the app will throw an exception if the previous statement fails to deliver return await func(dbSet!).ConfigureAwait(false); } private TResult ExecuteDbSetFunc<TResult>(Func<IMongoCollection<TDocument>, TResult> func) { var dbSet = GetDbSet(); // we know the app will throw an exception if the previous statement fails to deliver return func(dbSet!); } private IMongoCollection<TDocument> GetDbSet() { try { return context.GetCollection<TDocument>(typeof(TDocument).Name); } catch (Exception e) { logger.LogError(e, "Failed to get collection for entity", new { EntityType = typeof(TDocument).Name }); throw; } } }
Based on the above class, taking the GetByIdAsync method, I need to be able to mock the following:
- Given a TestDocument class, and a TestDocument repository, the mongo context returns an IMongoCollection of type TestDocument
- Given that collection I then need to create a verifiable setup that the method FindAsync is called with the expected Json filter
- That setup would also need to return an IAsyncCursor of type TestDocument
- Finally I needed to ensure that the IAsyncCursor delivered an expected document back out
First I set up my stubbed document class and repository:
public class TestDocumentRepository : DocumentRepositoryBase<TestDocument> { public TestDocumentRepository(IMongoContext context, ILogger logger) : base(context, logger) { } } public class TestDocument : IndexedEntityBase { }
Now I can set up my test with initial assertions. There are a lot of extension methods behind the FindAsync method and the one you need to setup has the following signature:
Task<IAsyncCursor<TProjection>> FindAsync<TProjection>( FilterDefinition<TDocument> filter, FindOptions<TDocument, TProjection> options = null, CancellationToken cancellationToken = default(CancellationToken));
There is no projection to TProjection is the same type of TestDocument, we are no setting any options so this can be set via the It.IsAny<> approach. The FilterDefinition<TDocument> is my biggest concern but, luckily, there is code within the MongoDb.Driver repo that can convert this to a json string. It makes sense to steal this code to use for our purpose here:
public static class FilterDefinitionExtensions { public static string RenderToJson<TDocument>(this FilterDefinition<TDocument> filter) { var serializerRegistry = BsonSerializer.SerializerRegistry; var documentSerializer = serializerRegistry.GetSerializer<TDocument>(); return filter.Render(documentSerializer, serializerRegistry).ToJson(); } }
Armed with this knowledge and tools, here’s my first pass:
// arrange var target = new TestDocumentRepository(context, logger); var testCollection = Mock.Of<IMongoCollection<TestDocument>>(); var id = Guid.NewGuid(); var findCollection = Mock.Of<IAsyncCursor<TestDocument>>(); var expected = new TestDocument(); var expectedJsonFilter = "{ \"_id\" : CSUUID(\"" + id + "\") }"; Mock.Get(context) .Setup(x => x.GetCollection<TestDocument>(typeof(TestDocument).Name)) .Returns(testCollection) .Verifiable(); Mock.Get(testCollection) .Setup(x => x.FindAsync( It.Is<FilterDefinition<TestDocument>>(filter => filter.RenderToJson().Equals(expectedJsonFilter)), It.IsAny<FindOptions<TestDocument, TestDocument>>(), default)) .Returns(Task.FromResult(findCollection)) .Verifiable(); // still needs to set up behaviour of the findCollection to return the expected document // act var actual = await target.GetByIdAsync(id).ConfigureAwait(false); // assert Mock.Verify(Mock.Get(context)); Mock.Verify(Mock.Get(testCollection)); // don't expect this to pass yet // Assert.Equal(expected, actual);
This works fine but, as per the comments, I recognise I’ve not yet completed the coverage as I need to get that final step of retrieving the document. You cannot simply mock the SingleOfDefaultAsync extension method so I needed to dive deeper into the implementation of IAsyncCursor.cs to find the following code:
public static TDocument SingleOrDefault<TDocument>(this IAsyncCursor<TDocument> cursor, CancellationToken cancellationToken = default(CancellationToken)) { using (cursor) { var batch = GetFirstBatch(cursor, cancellationToken); return batch.SingleOrDefault(); } } private static IEnumerable<TDocument> GetFirstBatch<TDocument>(IAsyncCursor<TDocument> cursor, CancellationToken cancellationToken) { if (cursor.MoveNext(cancellationToken)) { return cursor.Current; } else { return Enumerable.Empty<TDocument>(); } }
From this code I could deduce that I needed to mock the MoveNext method on my IAsyncCursor to return true and also mock the Current property to return an enumerable with a single item (my expected document):
Mock.Get(findCollection) .Setup(x => x.MoveNext(It.IsAny<CancellationToken>())) .Returns(true); Mock.Get(findCollection) .Setup(x => x.Current) .Returns(new[] { expected });
This then ensured I could uncomment my final assertion and verify the returned document was the expected one
Assert.Equal(expected, actual);
Determining json filter
What I found invaluable to me during this exercise was to start with a non specific FilterDefinition and then use a CallBack to determine what the json looked like so I could then specify the expectations within my setup. Purists may claim that is the tail wagging the dog but I can’t imagine many are able to translate anything more than the simplest of expressions or FilterDefinitions into json just by looking at it. This was how I worked out what json was being created (before replacing it with the intended test code):
Mock.Get(testCollection) .Setup(x => x.FindAsync( It.IsAny<FilterDefinition<TestDocument>>(), It.IsAny<FindOptions<TestDocument, TestDocument>>(), default)) .Callback((FilterDefinition<TestDocument> filter, FindOptions<TestDocument, TestDocument> options, CancellationToken ct) => { // stick a breakpoint here to find out what the json filter looks like var json = filter.RenderToJson(); }) .Returns(Task.FromResult(findCollection)) .Verifiable();
Returning a collection from IAsyncCursor
The approach I took to setup the IAsyncCursor to return my expected document for a GetById/ SingleOrDefault worked fine as the code ensured I got back a collection as long as MoveNext returned true.
Not unexpectedly, the code for returning a collection loops whilst MoveNext is true:
public static List<TDocument> ToList<TDocument>(this IAsyncCursor<TDocument> source, CancellationToken cancellationToken = default(CancellationToken)) { Ensure.IsNotNull(source, nameof(source)); var list = new List<TDocument>(); // yes, we are taking ownership... assumption being that they've // exhausted the thing and don't need it anymore. using (source) { // if I mock MoveNext to always return true this will never exit while (source.MoveNext(cancellationToken)) { list.AddRange(source.Current); cancellationToken.ThrowIfCancellationRequested(); } } return list; }
Seeing as I am setting up my expected collection, I should know the length of that collection and from this I can set up a callback to keep track of an index count and couple this with an Action<bool> to return true or false based on that index count:
int index = 0; var expected = new[] { new TestDocument(), new TestDocument() }; Mock.Get(context) .Setup(x => x.GetCollection<TestDocument>(typeof(TestDocument).Name)) .Returns(testCollection) .Verifiable(); Mock.Get(findCollection) .Setup(x => x.MoveNext(It.IsAny<CancellationToken>())) .Callback((CancellationToken ct) => { index++; }) .Returns(() => index < expected.Length);
Note you must return an Action<bool> and not simply use the index < expected.length as this would only be computed the once and you still end up in an infinite loop (#spokenfromexperience)!
As mentioned, the whole code can be found within the package repository on Github here. Feel free to check it out and suggest any improvements you think can be made.
2 Comments
Steve · 1st September 2020 at 4:42 pm
Thank you for writing this article, It really helped.
Fabiano · 21st June 2021 at 7:01 pm
Thanks a lot. You were a life saver :).