Skip to content Skip to sidebar Skip to footer
Home / How to Send Json String With Form Upload to Web Api

How to Send Json String With Form Upload to Web Api

Post a Comment

While working on an API that was built specifically for mobile clients, I ran into an interesting problem that I couldn't believe I hadn't found before. When working on a Rest API that deals exclusively in JSON payloads, how practice you upload images? Which naturally leads onto the side by side question, should a JSON API and then brainstorm accepting multipart form data? Is that non going to look weird that for every endpoint, we take JSON payloads, only then for this one nosotros accept a multipart form? Because we volition want to be uploading metadata with the image, we are going to take to read out formdata values too. That seems so 2000's! And then permit's see what nosotros can practise.

While some of the examples within this post are going to be using .NET Core, I retrieve this really applies to whatever language that is being used to build a RESTful API. So fifty-fifty if yous aren't a C# guru, read on!

Initial Thoughts

My initial thoughts sort of boiled down to a couple of points.

  • The API I was working on had been hardcoded in a couple of areas to really be forcing the whole JSON payload affair. Calculation in the ability to accept formdata/multipart forms would exist a little bit of work (and regression testing).
  • We had custom JSON serializers for things like decimal rounding that would somehow manually need to be washed for form data endpoints if required. We are even using snake_case equally property names in the API (dang iOS developers!), which would have to be done differently in the class data post.
  • And finally, is there any style to just serialize what would accept been sent nether a multi-function grade post, and include it in a JSON payload?

What Else Is Out At that place?

It really became articulate that I didn't know what I was doing. So like any good developer, I looked to copy. So I took a wait at the public API's of the three social media giants.

Twitter

API Doc : https://programmer.twitter.com/en/docs/media/upload-media/api-reference/post-media-upload

Twitter has 2 unlike ways to upload files. The first is sort of a "chunked" style, which I presume is because you tin can upload some pretty large videos these days. And a more simple mode for only uploading full general images, let'southward focus on the latter.

Information technology's a multi-office form, but returns JSON. Boo.

The very very interesting function about the API even so, is that it allows uploading the actual data in two ways. Either yous can upload the raw binary data as you typically would in a multipart form post, or you could actually serialise the file as a Base64 encoded cord, and send that every bit a parameter.

Base64 encoding a file was interesting to me because theoretically (And we nosotros will see afterwards, definitely), we can ship this string data any mode we similar. I would say that of all the C# SDKs I looked at, I couldn't find any really using this Base64 method, so there weren't any peachy examples to go off.

Another interesting point about this API is that you are uploading "media", and then at a later date attaching that to an actual object (For example a tweet). And so if you wanted to tweet out an image, it seems similar you would (correct me if I'm wrong) upload an paradigm, become the ID returned, and then create a tweet object that references that media ID. For my use case, I certainly didn't want to do a 2 pace process like this.

LinkedIn

API Medico : https://programmer.linkedin.com/docs/guide/v2/shares/rich-media-shares#upload

LinkedIn was interesting because it'southward a pure JSON API. All data POSTs contain JSON payloads, similar to the API I was creating. Wouldn't you lot guess it, they use a multipart grade data as well!

Similar to Twitter, they as well take this concept of uploading the file first, and attaching it to where you actually want information technology to end up second. And I totally get that, it'south just non what I want to practise.

Facebook

API Doc : https://developers.facebook.com/docs/graph-api/photo-uploads

Facebook uses a Graph API. So while I wanted to take a look at how they did things, and then much of their API is not really relevant in a RESTful world. They do use multi-part forms to upload data, but it's kinda hard to say how or why that is the case,. Also at this bespeak, I couldn't go my mind off how Twitter did things!

And then Where Does That Leave Us?

Well, in a weird way I think I got what I expected, That multipart forms were well and truly alive. It didn't seem like there was any nifty innovation in this area. In some cases, the employ of multipart forms didn't look so fell considering they didn't need to upload metadata at the same fourth dimension. Therefore simply sending a file with no attached data didn't await so out of place in a JSON API. However, I did desire to transport metadata in the aforementioned payload as the image, not take information technology equally a two step process.

Twitter'due south employ of Base64 encoding intrigued me. Information technology seemed like a pretty skilful option for sending information across the wire irrespective of how you were formatting the payload. Y'all could transport a Base64 string as JSON, XML or Grade Data and it would all exist handled the same. It's definitely proof of concept time!

Base64 JSON API POC

What we want to do is merely test that nosotros can upload images every bit a Base64 string, and we don't take any major issues inside a super simple scenario. Notation that these examples are in C# .Internet Core, but once again, if y'all are using whatsoever other language information technology should be fairly elementary to translate these.

First, we need our upload JSON Model. In C# it would be :

public class UploadCustomerImageModel { 	public string Description { get; set; } 	public string ImageData { get; set; } }

Not a whole lot to it. Simply a description field that tin can be freetext for a user to describe the prototype they are upload, and an imagedata field that will concur our Base64 string.

For our controller :

[HttpPost("{customerId}/images")] public FileContentResult UploadCustomerImage(int customerId, [FromBody] UploadCustomerImageModel model) { 	//Depending on if you want the byte assortment or a memory stream, yous can utilise the below.  	var imageDataByteArray = Catechumen.FromBase64String(model.ImageData); 	//When creating a stream, you need to reset the position, without information technology you will come across that you lot e'er write files with a 0 byte length.  	var imageDataStream = new MemoryStream(imageDataByteArray); 	imageDataStream.Position = 0; 	//Go and do something with the actual data. 	//_customerImageService.Upload([...]) 	//For the purpose of the demo, we return a file so we tin ensure it was uploaded correctly.  	//Merely otherwise yous can just return a 204 etc.  	return File(imageDataByteArray, "image/png"); }

Once more, fairly damn simple. We accept in the model, and so C# has a smashing mode to convert that cord into a byte assortment, or to read information technology into a memory stream. Also note that as we are just building a proof of concept, I repeat out the prototype data to brand sure that it's been received, read, and output similar I wait information technology would, just not a whole lot else.

At present let's open up up postman, our JSON payload is going to look a bit similar :

{ 	"description" : "Test upload of an prototype",  	"imageData" : "/9j[...]" }

I've obviously truncated imagedata down hither, merely a super elementary tool to plow an image into a Base64 is something like this website here. I would too note that when you ship your payload, it should exist without the data:image/jpeg;base64, prefix that yous sometimes meet with online tools that convert images to strings.

Hit send in Postman and :

Great! So my image upload worked and the picture of my cat was echo'd back to me! At this point I was actually kinda surprised that information technology could be that like shooting fish in a barrel.

Something that became very axiomatic while doing this though, was that the payload size was much larger than the original image. In my case, the image itself is 109KB, only the Base64 version was 149KB. Then virtually 136% of the original prototype. In having a quick search around, it seems expected that a Base64 version of a file would exist about 33% bigger than the original. When it comes to larger files, I think less well-nigh sending 33% more across the wire, but more the fact of reading the file into memory, and then converting information technology into a huge string, and then writing that out… It could cause a few issues. Only for a few basic images, I'm comfortable with a 33% increase.

I will also notation that there was a few code snippets around for using BSON or Protobuf to do the aforementioned affair, and may actually cut down the payload size substantially. The mentality would be the same, a JSON payload with a "stringify'd" file.

Cleaning Up Our Code Using JSON Converters

One thing that I didn't similar in our POC was that we are using a string that almost certainly will be converted to a byte assortment every single time. The dandy thing most using a JSON library such as JSON.net in C#, is that how the client sees the model and how our backend code sees the model doesn't necessarily have to exist the exact same. So permit's see if we tin can plow that string into a byte array on an API POST automagically.

Get-go we need to create a "Custom JSON Converter" form. That lawmaking looks like :

public class Base64FileJsonConverter : JsonConverter { 	public override bool CanConvert(Blazon objectType) 	{ 		return objectType == typeof(string); 	} 	 	public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) 	{ 		return Convert.FromBase64String(reader.Value every bit cord); 	} 	//Because nosotros are never writing out every bit Base64, we don't need this.  	public override void WriteJson(JsonWriter author, object value, JsonSerializer serializer) 	{ 		throw new NotImplementedException(); 	} }

Fairly simple, all we are doing is taking a value and converting it from a string into a byte assortment. Also notation that we are just worried virtually reading JSON payloads here, we don't intendance about writing as we never write out our epitome as Base64 (yet).

Adjacent, we had dorsum to our model and we employ the custom JSON Converter.

public class UploadCustomerImageModel {     public string Description { become; set; }     [JsonConverter(typeof(Base64FileJsonConverter))]     public byte[] ImageData { get; gear up; } }

Note we also change the "type" of our ImageData field to a byte array rather than a cord. And then fifty-fifty though our postman test will nevertheless send a string, past the time information technology actually gets to the states, it will be a byte array.

We will also need to modify our Controller lawmaking also :

[HttpPost("{customerId}/images")] public FileContentResult UploadCustomerImage(int customerId, [FromBody] UploadCustomerImageModel model) {     //Depending on if you desire the byte assortment or a retentivity stream, you can use the beneath.      //THIS IS NO LONGER NEEDED Equally OUR MODEL At present HAS A BYTE Assortment     //var imageDataByteArray = Convert.FromBase64String(model.ImageData);     //When creating a stream, y'all need to reset the position, without it yous will run into that y'all always write files with a 0 byte length.      var imageDataStream = new MemoryStream(model.ImageData);     imageDataStream.Position = 0;     //Go and do something with the actual information.     //_customerImageService.Upload([...])     //For the purpose of the demo, we return a file so nosotros can ensure information technology was uploaded correctly.      //But otherwise you can simply return a 204 etc.      return File(model.ImageData, "image/png"); }

So it becomes even simpler. We no longer demand to bother handling the Base64 encoded string anymore, the JSON converter will handle it for us.

And that'south it! Sending the exact same payload volition nonetheless work and nosotros accept one less piece of plumbing to practise if we make up one's mind to add more endpoints to take file uploads. Now you are probably thinking "Yeah but if I add together in a new endpoint with a model, I still need to remember to add the JsonConverter attribute", which is true. Simply at the same time, it ways if in the future you decide to swap to BSON instead of Base64, you aren't going to have to go to a tonne of places and work out how you are handling the incoming strings, information technology's all in one handy place.

floresfrompont.blogspot.com

Source: https://dotnetcoretutorials.com/2018/07/21/uploading-images-in-a-pure-json-api/

Post a Comment for "How to Send Json String With Form Upload to Web Api"