DevTut

# Connecting with Microsoft Cognitive Services

# Connecting with Microsoft Cognitive Services

In this example you will learn how to use Microsoft Cognitive Services with Xamarin iOS mobile application. We will use Computer Vision API to detect what is in the picture.

Once you create Xamarin.iOS project please add below NuGet package to the project:

https://www.nuget.org/packages/Microsoft.ProjectOxford.Vision/ (opens new window)

With this library we will be able to utilize Cognitive Services in our iOS app. I assume that you have Microsoft account already registered to use it and you have enabled Computer Vision Api like on the screen below: enter image description here (opens new window)

Once you click "Subscribe" at the bottom Api Key will be generated:

enter image description here (opens new window)

Now we can start configuring access to Cognitive Services from the iOS app. Firstly we need to get some picture for the analysis. To do it we can use Xamarin Media Component available below: https://components.xamarin.com/view/mediaplugin (opens new window)

Once it is successfully installed let's create simple UI with the image and button to select picture from the gallery. Size of the controls is up to you.

Open Main.storyboard and add UIImageView and UIButton controls do default ViewController. Add them names: "SelectedPictureImageView" and "SelectButton":

enter image description here (opens new window)

Now we should add "Touch Up Inside" event handler to handle image selection:

partial void SelectButtonClick(UIButton sender)
{
    selectImage();
}

async void selectImage()
{
    var selectedImage = await CrossMedia.Current.PickPhotoAsync();
    SelectedPictureImageView.Image =  new UIImage(NSData.FromStream(selectedImage.GetStream()));
}

Now we would like to display analysis information once Cognitive Services returns the information. Add label below the button called "AnalysisLabel": enter image description here (opens new window)

It is time to connect Computer Vision API!

To get information about selected picture add below method. Remember to paste you API Key!

async Task analyseImage(Stream imageStream)
{
    try
    {
        VisionServiceClient visionClient = new VisionServiceClient("<<YOUR API KEY HERE>>");
        VisualFeature[] features = { VisualFeature.Tags, VisualFeature.Categories, VisualFeature.Description };
        var analysisResult = await visionClient.AnalyzeImageAsync(imageStream, features.ToList(), null);
        AnalysisLabel.Text = string.Empty;
        analysisResult.Description.Tags.ToList().ForEach(tag => AnalysisLabel.Text = AnalysisLabel.Text + tag + "\n");
    }
    catch (Microsoft.ProjectOxford.Vision.ClientException ex)
    {
        AnalysisLabel.Text = ex.Error.Message;
    }
}

Now you can invoke it in "selectImage" method:

async void selectImage()
{
    var selectedImage = await CrossMedia.Current.PickPhotoAsync();
    SelectedPictureImageView.Image =  new UIImage(NSData.FromStream(selectedImage.GetStream()));
    await analyseImage(selectedImage.GetStream());
}

Once you select image, Microsoft Cognitive Services will analyze it and return the result:

enter image description here (opens new window)

Remember that you image cannot be too large - in this case you will receive information like below:

enter image description here (opens new window)

There are many other services which you can try to use. Please refer to the official documentation (link attached) to discover more.

# Remarks

In this example we used Microsoft.ProjectOxford.Vision NuGet package: https://www.nuget.org/packages/Microsoft.ProjectOxford.Vision/ (opens new window)

To read more about Microsoft Cognitive Services please refer to the official documentation: https://www.microsoft.com/cognitive-services/en-us/computer-vision-api (opens new window)

Please find uploaded sample on my GitHub: https://github.com/Daniel-Krzyczkowski/XamarinIOS/tree/master/XamariniOS_CognitiveServices (opens new window)

I also attach link to my blog where I presented how to use Cognitive Services with Xamarin Forms application: http://mobileprogrammer.pl (opens new window)

← Concurrent Programming in Xamarin.iOS →