Join Chander Dhall for an in-depth discussion in this video Add JSON from file, part of Cosmos DB: Import, Manipulate, Index and Query.
- [Instructor] So as we just saw that we could take any object which is a JSON object and then convert it over to a C# object and then have that document imported to DocumentDB or Cosmos DB. However, most of the times what we're going to do is take something like this, which is a big array of JSON data, and push it as it is programmatically. Now we have two different ways to do this.
In the previous course on NoSQL Development with DocumentDB in Azure, we saw how we could use the data migration tool, but a lot of times we have to do this programmatically. So what I'm going to show you is another technique where you could take the JSON data as it is from a file or JSON data in memory and then send it over to your Azure instance of Cosmos DB. So we'll go ahead and remove these methods because we don't need them anymore, and remove this particular call, and keep everything as it is.
The next thing we're going to do is create that method we just talked about. Say static, again, async and it returns the same thing, which is a Cosmos DB document, and then say CreateCollectionFromJson as it is, without creating a typed object, and again, what we need is, the collectionId, the databaseId, and of course, one good thing to have would be the name of the file.
So we can say string filePath, and then the fourth thing we would need is a Stream that is a Stream that has the entire JSON. So what's the first thing? We need the collection, and we'll do it the same way we've been doing it, ReadDocumentCollectionAsync, and then create the URI from UriFactory, CreateDocumentCollectionUri, and in this case, we're going to pass in the databaseId, and the collectionId.
So once we have the collection, we have the resource link, which comes from collection.resource. Next, we'll have document, create an instance of document, and say await, similar code as what we did in the last module, CreateDocumentAsync, and then we have collection.resource, and the reason I don't see intellisense is because there was a typo here.
Once I change that, I will get the collection back, and say .Resource.SelfLink. So now that I have the document, there's another option here which is reload the document from the stream that we just passed in, and then return the document. Now this way, what we could do is CreateCollectionFromJson, and the major difference between the two approaches is the stream.
In this case, what we're doing is we're taking the stream, which is a JSON stream, and passing it as it is to Cosmos DB, and what's the next thing? This particular function needs to be called in another function. Again it's going to be static async, and it doesn't really have anything to return so it's going to be a void FromFile, and then it takes in few things that we need to make this work.
As you can see in this case, we never use the filePath, so we don't really have to pass this filePath in right here, we can only deal with the filePath in this particular function, which is AddJsonFromFile. Very similar approach, like what we did in the previous module, we have StreamReader, and again, we're using the using statement just to make sure we dispose of the StreamReader after we're done with it.
So you have string line, and then you have the same exact logic, which we used last time, which was file.ReadLine, and as long as it can read a file, continue the while loop, and next what we need to do is take this string and convert it into a byteArray. We're going to use Encoding.UTF8, and then do GetBytes(), and as you can see, it takes few different things including a character array on a string, and since line is a string, this should work.
Once we have all this, we're going to use another using statement and create a MemoryStream because we need the stream to pass it over to this particular function, which is CreateCollectionFromJson, and we pass in the byteArray to the stream, and then call that particular function, and now we have the databaseId, then we have the collectionId, and then we have the stream which we just created.
Keep in mind, it's an Async function, so we'll have to do Wait() right here and that's it. Now all we need to do is call AddJsonFromFile() and then pass in the values. So in this case, we can say Cazton, which is the databaseId, and then we have the collectionId. We can go in and use Test in this case, because we just want to see different data coming in, last time we used Company, this time we can use Test, and then of course filePath, and since it's in the same directory, we don't have to qualify this with any sort of path but if it was inside another directory, you would use the (mumbles) path of that particular file.
Ctrl + F5, as you can see, we did not use a console directing statement so this will not show anything, and something is going on in the background, because it's a lot of data that is going over to Azure. In the meanwhile, we can go back to the portal, and go to Document Explorer, and select Test, and you can see that the data has started making it into this particular collection, which is Test, and now you can see that we have lot more data than we had last time, and it's not that we cannot recreate this with an entity in C# like what we did in case of Company, we just happened to use five properties, and all the five properties were showing up, but as you can see, this is a lot of data and that's because it's taking the JSON as it is, and adding that data as it is to Azure Cosmos DB.
The only difference you'll see is addition of an id that's created by Azure Cosmos DB, and then you can pick a category code called news and you can see that this data is a lot of data. So that works.
- Data import scenarios
- Creating a database
- Creating a partitioned collection
- Data manipulation
- Importing documents with a stored procedure
- User-defined functions
- Excluding indexing at a document level
- Range indexing on strings
- Querying with SQL parameters
- Range operations