Intel Galileo Windows Application To Azure Storage

Intel Galileo on Windows platform currently supports only C++ project in Visual Studio, so if you want to connect your Galileo project to Azure Storage, you will need to code it in C++ and use only C++ libraries. And..there is some good and bad news about that

The good news:

  • there is a C++ SDK availible that enables calling REST services. It is C++ REST SDK (code name Casablanca)
  • there is a C++ Azure Storage library, which is using C++ REST SDK and provides C++ classes for the communication with Storage services on Azure

The bad news:

  • both libraries won’t work on Galileo without some changes/workarounds.
  • standard Windows libraries don’t work exactly the same way on standard workstation Windows and on Galileo. Because of this differences, calling Azure services from Galileo wan’t work at all.

But, the good news..there is a workaround, that I’m going to explain, which enable calling Azure services directly from Galileo. In this post I will discribe both C++ REST and Azure Storage library and explain what changes are needed to make it works on Galileo Board.

Casablanca Project

Casablanca is a code name of the C++ REST SKD library project. It’s a library from Microsoft published as an open source project on CodePlex. https://casablanca.codeplex.com

Casablanca allows you to write native code to access REST services and uses an asynchronous approach to consuming HTTP and JSON based services.The API surface of C++ Rest SDK is platform-agnostic, and it already supports many platforms including iOS and Android. And of course, you can use it on Windows. But what about Windows on Galileo? At this moment you can use this SDK in Galileo project only with some modifications. Currently only version 2.2. can be used on Galileo. Instructions how to use it are here: http://ms-iot.github.io/content/Casablanca.htm

As you can read on this location, you should build a project by yourself using an important change in project properties:

  • Set Enable Enhanced Instruction Set = No Enhanced Instructions (/arch:IA32) for both Debug and Release

Why is that necessary? Normally, when compiling a Win32 application for a desktop PC, we will compile against Win32 libraries contained in C:\Windows\System32.  When targeting the Intel Galileo board you will notice that the default Intel Galileo Wiring App template links against a single library, mincore.lib. So, we need to compile against mincore.lib in order to obtain code capable of running on the Galileo as the mappings for System and Win32 functions are completely different in Windows for IoT and contained in this particular lib. GalileoAppLinkerProperties

We should also consider the hardware present on the Galileo board itself.  Intel outfits the board with an Intel® Quark™ SoC X1000 application processor, a 32-bit, single-core, single-thread, Intel® Pentium® processor instruction set architecture (ISA)-compatible, operating at speeds up to 400 MHz.  This processor does not support enhanced instruction set including SSE, SSE2, AVX, or AVX 2. And that’s the reason we should ensure all code is compiled with the /arch:IA32 compiler flag GalileoCodeGenerationProperties

Using these changes you should create your own Casablanca 2.2 package that will be used in Azure Storage C++ library.

Azure Storage Client Library for C++

The Azure Storage Client Library for C++ allows you to build applications against Microsoft Azure Storage. For an overview of Azure Storage, see Introduction to Microsoft Azure Storage.

Currently the Azure Storage Client Library for C++ library is a CTP (Community Technology Preview) release. Current version is 0.4.0. And if you just take it to your Galileo project, it won’t work. But, the good news, this is an open source project and with some modifications, it works just fine.

The Project is on GitHub https://github.com/Azure/azure-storage-cpp

And you can easily get the code to your workspace

git clone https://github.com/Azure/azure-storage-cpp.git

After that, you should change the depending Casablanca project version to your local 2.2 version of the C++ REST SDK and compile a solution. If you already prepare your own Casablanca 2.2 version as described in previous section, you can open the solution “Microsoft.WindowsAzure.Storage.v120.sln”, go to Tools “NuGet Package Manager-> Package Manager Console” and type:

Install-Package cpprestsdk -Version 2.2.0

After you build the solution, you should take the compiled dll and lib and take it to your Galileo project.

Let’s see how it works:

    1. Create Galileo Wiring App Project
    2. Install NuGet Package CPP Rest SDK Version 2.2
      • Install-Package cpprestsdk -Version 2.2.0
    3. Manually add Azure Storage Client Library for C++ in your project
      • Copy was and wascore folders to your project folder
      • Copy wastorage.lib to your project folder
      • Copy wastorage.dll to your output folder on your desktop. After deploy on Galileo, you should manually copy this dll to Galileo in output folder
      • Add library to your project settingsimage
    4. Add  following lines at the beginning of your main.cpp
      #include <cpprest/http_client.h>
      #include <cpprest/filestream.h>
      
      #include "was\storage_account.h"
      #include "was\table.h"
      
      #include "stdafx.h"
      #include "arduino.h"
      
      using namespace utility; // Common utilities like string conversions
      using namespace web; // Common features like URIs.
      using namespace web::http; // Common HTTP functionality
      using namespace web::http::client; // HTTP client features
      using namespace concurrency::streams; // Asynchronous streams
      
      using namespace azure::storage;
    5. Add following lines in void setup()
      void setup()
      {
      storage_credentials creds(L"{storage account name}", L"{azure key}");
      
      web::http::uri uriEndPoint(L"https://{azure storage name}.table.core.windows.net");
      storage_uri uritable(uriEndPoint);
      cloud_table_client table_client(uritable, creds);
      table = table_client.get_table_reference(L"{storage table name}");
      }
    6. Add code to send data to Azure Storage using following code
      void AddDataToStarage(utility::string_t partition_key, utility::string_t row_key, int val1, int val2)
      {
      table_entity entity(partition_key, row_key);
      auto& properties = entity.properties();
      properties.reserve(2);
      properties[U("{storage column1 name}")] = entity_property(val1);
      properties[U("{storage column2 name}")] = entity_property(val2);
      auto operation = table_operation::insert_entity(entity);
      try
      {
      auto result = table.execute(operation);
      }
      catch (azure::storage::storage_exception& e)
      {
      auto s = e.what();
      Log(s);
      }}
    7. Now you can call this method in setup or in loop method of your Galileo Wiring App

This instruction doesn’t describe how to create Storage Table on Azure. If you need information how to create Azure Storage and add tables to your storage, you can find it here http://azure.microsoft.com/en-us/documentation/articles/storage-create-storage-account/

Now let’s start this application locally. It should work correctly.

The show begins when you try to run it on Galileo board. Let’s start a romore debug session. Don’t forget to copy wastorage.dll to your Galileo deploy folder (“\\mygalileo\c$\test”).

When you run the remote session, you will probably see something like this: IoT VS Debug Output

Entry point not found doesn’t really explains much. The problem is that in some depending dll, some method is missing. To enable more diagnostic output, you can change registry key GlobalFlag=2 under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager.

After that you’ll see that following methods is missing UnregisterClassW. This method is defined in User32.dll and if you compare User32.dll in you workstation Windows with the one in Galileo, you’ll see that these files are pretty much different. The Galileo version doesn’t have a lot of method which are defined in standard Windows version. Our Application need only one Method form this dll and it is UnregisterClassW, which is in our scenario not very important, but without this method the application can’t run at all. So let see hot to fix/workaround this problem.

Galileo Workaround with Azure Storage Client Library

First we need to remove dependency on User32 library in Microsoft.WindowsAzure.Storage project and we will also remove dependency to some other default libraries: image

In Additional options you should add:

-d2:-nolock /NODEFAULTLIB:ole32.lib /NODEFAULTLIB:kernel32.lib /NODEFAULTLIB:advapi32.lib /NODEFAULTLIB:user32.lib 

And build the solution “Microsoft.WindowsAzure.Storage.v120” again. It doesn’t solve all problems. Now you will get a new problem:

error LNK2001: unresolved external symbol _UnregisterClassW@8

This error is obvious. We call this method somewhere deeply and it’s not implemented, because we remove user32.lib from our project.

To fix the problem, I’ve made an ugly workaround, but it works perfectly. I defined a dummy method in my version of Microsoft.WindowsAzure.Storage.v120 project:

BOOL WINAPI UnregisterClassW(
    _In_      LPCTSTR lpClassName,
    _In_opt_  HINSTANCE instance
    )
{
    UNREFERENCED_PARAMETER(lpClassName);
    UNREFERENCED_PARAMETER(instance);
    return TRUE;
};

It can be defined in any c++ file in project.

Compile the project and copy new version on wastorage.lib and dll to you Galileo project. Compile your Galileo project, copy wastorage.dll manually to your test folder on Galileo and start the application on Galileo. The “entry point not found” error is gone. Application is running!

But, we have another problem now. The Azure storage returns an error:

The remote server returned an error: (403) Server failed to authenticate the request.

Date Format in REST Azure Storage call from Galileo

The Azure storage returns 403 error only if it’s called from Galileo. It works perfect when it’s called from a standard workstation. If you compare POST command (using visual studio, fiddler or some other diagnostic tool) you will see the difference in date (x-ms-date) format in header.

All authenticated requests must include the Coordinated Universal Time (UTC) timestamp for the request. You can specify the timestamp either in the x-ms-date header, or in the standard HTTP/HTTPS Date header. If both headers are specified on the request, the value of x-ms-date is used as the request’s time of creation.

The storage services ensure that a request is no older than 15 minutes by the time it reaches the service. This guards against certain security attacks, including replay attacks. When this check fails, the server returns response code 403 (Forbidden).

I our case the problem is in format.  From unknown reason (probably a bug in Galileo Windows), date Format in Galileo differs from the expected date format in Azure REST header. So there is also a workaround for this problem. In method shared_key_authentication_handler::sign_request (azure-storage-cpp-master\Microsoft.WindowsAzure.Storage\src\authentication.cpp) change the code to avoid system formatting of x-ms-date in REST signing:

void shared_key_authentication_handler::sign_request(web::http::http_request& request, operation_context context) const
{
	web::http::http_headers& headers = request.headers();
	utility::datetime utcnow = utility::datetime::utc_now();
	int status;

	ULARGE_INTEGER largeInt;
	largeInt.QuadPart = utcnow.to_interval();

	FILETIME ft;
	ft.dwHighDateTime = largeInt.HighPart;
	ft.dwLowDateTime = largeInt.LowPart;

	SYSTEMTIME systemTime;
	if (!FileTimeToSystemTime((const FILETIME *)&ft, &systemTime))
	{
		throw utility::details::create_system_error(GetLastError());
	}

	static const char *DAY_NAMES[] =
	{ "Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat" };
	static const char *MONTH_NAMES[] =
	{ "Jan", "Feb", "Mar", "Apr", "May", "Jun",
	"Jul", "Aug", "Sep", "Oct", "Nov", "Dec" };

	struct tm tm;
	tm.tm_sec = systemTime.wSecond;
	tm.tm_min = systemTime.wMinute;
	tm.tm_hour = systemTime.wHour;
	tm.tm_mday = systemTime.wDay;
	tm.tm_mon = systemTime.wMonth -1;
	tm.tm_year = systemTime.wYear -1900;
	tm.tm_wday = systemTime.wDayOfWeek;

	mktime(&tm);
	char buf[30];

	strftime(buf, 30, "---, %d --- %Y %H:%M:%S GMT", &tm);
	memcpy(buf, DAY_NAMES[tm.tm_wday], 3);
	memcpy(buf + 8, MONTH_NAMES[tm.tm_mon], 3);

	std::wostringstream outStream;

	outStream << buf; 	utility::string_t utcnows =outStream.str(); 	headers.add(ms_header_date, utcnows); 	if (m_credentials.is_shared_key()) 	{ 		utility::string_t string_to_sign = m_canonicalizer->canonicalize(request, context);
            
		if (core::logger::instance().should_log(context, client_log_level::log_level_verbose))
		{
			utility::string_t with_dots(string_to_sign);
			std::replace(with_dots.begin(), with_dots.end(), U('\n'), U('.'));
			core::logger::instance().log(context, client_log_level::log_level_verbose, U("StringToSign: ") + with_dots);
		}

		utility::ostringstream_t header_value;
		header_value << m_canonicalizer->authentication_scheme() << U(" ") << m_credentials.account_name() << U(":") << calculate_hmac_sha256_hash(string_to_sign, m_credentials);

		headers.add(web::http::header_names::authorization, header_value.str());
	}
}

After this change, repeat the known steps: compile library, copy lib/dll to your application and to Galileo folder, compile and run the application remotly and everything should work OK.

I wish you happy coding with Galileo! And yet it moves!

image

10 thoughts on “Intel Galileo Windows Application To Azure Storage

  1. Pooja
    February 4, 2015 at 10:54 pm

    Hello sir,
    I am new to IoT, and struggling to create a small weather station for my project.
    Can you please tell me how to send all the sensors values to Azure with windows on Galileo.
    Thanks

    1. kardum
      February 11, 2015 at 9:25 am

      Hi!
      If you already have weather station information, you just need to send it to Azure Storage. I did it using Azure Storage Client Library for C++ as described in this blog post. Soon, I will publish my new blog post about similar scenario. How to collect temperature and humidity using DHT11 sensor and send it to Azure.

      1. Steve
        February 12, 2015 at 10:43 pm

        I am also waiting for your blog about weather station. When you are going to post that?

      2. Pooja
        February 12, 2015 at 11:36 pm

        Not working for me . I am getting lots of error.

        1. kardum
          February 13, 2015 at 8:21 am

          What kind of errors? Compile-time, linking or run-time errors? Can you copy/paste it? Just one or few of them. Maybe I can help you?

          1. Pooja
            February 13, 2015 at 7:34 pm

            When I tried to run the sample code after building the project as described here https://ms-iot.github.io/content/Casablanca.htm , I am getting these errors-

            Galileo Wiring App7.exe’ (Win32): Loaded ‘C:\Windows\System32\IPHLPAPI.DLL’. Cannot find or open the PDB file.
            ‘Galileo Wiring App7.exe’ (Win32): Loaded ‘C:\Windows\System32\WINNSI.DLL’. Cannot find or open the PDB file.
            ‘Galileo Wiring App7.exe’ (Win32): Loaded ‘C:\Windows\System32\DNSAPI.dll’. Cannot find or open the PDB file.
            ‘Galileo Wiring App7.exe’ (Win32): Loaded ‘C:\Windows\System32\Fwpuclnt.dll’. Cannot find or open the PDB file.
            Received response status code:200
            First-chance exception at 0x77330F63 in Galileo Wiring App7.exe: Microsoft C++ exception: _arduino_quit_exception at memory location 0x0070F9DC.
            The thread 0x80 has exited with code 0 (0x0).
            The thread 0x7b0 has exited with code 0 (0x0).
            The thread 0x5b8 has exited with code 0 (0x0).
            The thread 0x5d0 has exited with code 0 (0x0).
            The program ‘[60] Galileo Wiring App7.exe’ has exited with code 0 (0x0).

            This is some of many errors I am getting and when I tried to build the complete project as you suggested I am getting these errors

            ‘Galileo Wiring App1.exe’ (Win32): Loaded ‘C:\test\Galileo Wiring App1.exe’. Symbols loaded.
            ‘Galileo Wiring App1.exe’ (Win32): Loaded ‘C:\Windows\System32\ntdll.dll’. Cannot find or open the PDB file.
            ‘Galileo Wiring App1.exe’ (Win32): Loaded ‘C:\Windows\System32\kernel32.dll’. Cannot find or open the PDB file.
            ‘Galileo Wiring App1.exe’ (Win32): Loaded ‘C:\Windows\System32\kernel32legacy.dll’. Cannot find or open the PDB file.
            ‘Galileo Wiring App1.exe’ (Win32): Loaded ‘C:\Windows\System32\KernelBase.dll’. Cannot find or open the PDB file.
            ‘Galileo Wiring App1.exe’ (Win32): Loaded ‘C:\test\cpprest120d_2_2.dll’. Cannot find or open the PDB file.
            ‘Galileo Wiring App1.exe’ (Win32): Loaded ‘C:\test\wastorage.dll’. Symbols loaded.
            ‘Galileo Wiring App1.exe’ (Win32): Loaded ‘C:\Windows\System32\msvcp120d.dll’. Cannot find or open the PDB file.
            ‘Galileo Wiring App1.exe’ (Win32): Loaded ‘C:\Windows\System32\msvcr120d.dll’. Cannot find or open the PDB file.
            ‘Galileo Wiring App1.exe’ (Win32): Loaded ‘C:\Windows\System32\winhttp.dll’. Cannot find or open the PDB file.
            ‘Galileo Wiring App1.exe’ (Win32): Loaded ‘C:\Windows\System32\HTTPAPI.dll’. Cannot find or open the PDB file.
            ‘Galileo Wiring App1.exe’ (Win32): Loaded ‘C:\Windows\System32\bcrypt.dll’. Cannot find or open the PDB file.
            ‘Galileo Wiring App1.exe’ (Win32): Loaded ‘C:\Windows\System32\WS2_32.dll’. Cannot find or open the PDB file.
            The program ‘[1316] Galileo Wiring App1.exe’ has exited with code -1073741515 (0xc0000135) ‘A dependent dll was not found’.

          2. Pooja
            February 13, 2015 at 7:38 pm

            It will be so helpful if you can share the casablanca dll and other source code.

  2. Steve
    February 12, 2015 at 10:46 pm

    Can you please also include the source code for this i.e. the final project?

    1. kardum
      February 13, 2015 at 8:18 am

      In the next blog post about temp sensor reading, I will include a complete source code, including using of Azure Storage Client Library for C++. Hopefully I’ll blog about that next week.

  3. PG
    March 10, 2015 at 2:34 pm

    Is it possible to provide the wastorage.dll and cpprest120d_2_2.dll that you have re-built?

Leave a Reply

Your email address will not be published. Required fields are marked *