Japan SfB サポートチームです。
現在、Skype for Business for iOS を利用いただいているお客様では、
フェデレーション認証を実施する際、AD FS サーバーに送られる User-Agent を基に、
トークンの発行を許可する設定を行っていただいているお客様がいらっしゃいます。
これまでは、クレームルールを構成する際、User-Agent の列では、”Lync Mobile” のエージェント名が使われておりましたが、今後、随時以下に変更される状況を確認いたしました。
iPhoneLync
iPadLync
クレームルールをご調整いただき “Lync Mobile” と合わせまして、”iPhoneLync”, “iPadLync” も追加していただけますようお願いいたします。
これらの機能は、先週リリースされた Windows Insider Preview のビルド 15019 からデフォルトで有効になっており、Windows 10 Windows 10 Creators Update から安定したリリースになります。
マイクロソフト Edge は、EdgeHTML 13 (windows 10 バージョン 1511) で ORTC 開始のサポートを導入し、Edge でのリアルタイム通信の初期基盤を提供しています。WebRTC 1.0 API のサポートとの我々の優先順位は、既存の web サイト上のレガシー実装と相互運用性を提供することで、他のブラウザで以前にデプロイされた WebRTC API と同じように活用します。
私たちの WebRTC 1.0 API の実装は、2015 年頃の WebRTC オブジェクト モデルを追加する前の W3C WebRTC-PC API のサブセットに基づき、ピアツーピア音声およびビデオをサポートします。この実装はレガシの相互運用性 (WebRTC.org ソースコードの初期バージョンから構築されたモバイルアプリケーションを含む) に重点を置いているため、このリリース以降のネイティブ WebRTC 1.0 API をさらに更新する予定はありません。
Microsoft Edge RTC スタックの最も高度な機能を利用するには、とくに、個々のトランスポート、送信者、および受信者のオブジェクトを直接制御することが望ましい場合やグループのオーディオおよびビデオ通話を設定する場合には、ORTC API を考慮することをお勧めします。
現在の WebRTC 1.0 API を使用してマルチストリームや同時放送などのオブジェクトや高度な機能をサポートする必要がある場合は、Microsoft Edge をサポートする adapter.js ライブラリをお勧めします。
H.264/AVC と VP8 ビデオ コーデックは、Microsoft Edge RTC スタックでサポートされており、これは現在 Microsoft Edge とその他のメジャーな WebRTC ブラウザーおよび RTC サービスで、ビデオ通信などが相互運用が可能であることを意味しています。我々は、H.264/AVC と VP8 の両方のビデオコーデックのための次の輻輳制御と堅牢性のメカニズムを実装しています:
これらの機能は、ORTC API とネイティブ WebRTC 1.0 API の両方で利用できますので、API とビデオコーデックの決定を個別に行うことができます。
Edge の H.264/AVC の実装では、エンコーダーとデコーダーの両方でハードウェア オフロードをサポートしますが、VP8 は純粋にソフトウェアで実装されているので、結果として高い消費電力と CPU 負荷を示す場合があります。VP8 アプリケーションを使用する場合は、許容範囲のパフォーマンスを確保するためローエンド デバイス上でのテストをお勧めします。
リアルタイムコミュニケーションのロードマップを引き続き示していく中で、次の優先事項は、W3C Screen Capture 仕様のサポートとエンタープライズシナリオのサポートの強化です。作業の進行に伴い、更新とプレビュービルドの共有を楽しみにしています。一方、我々は WebRTC と私たちの現在の RTC のロードマップにあなたのフィードバックを楽しみにしています。今日、プレビュー ビルドで WebRTC 1.0 を試すことができ、バグが発生した場合には Twitter @MSEdgeDev、または下のコメント欄で経由で、Microsoft Edge Platform の問題についてのフィードバックを共有することができます。
― Shijun Sun, Principal Program Manager, Microsoft Edge
― Bernard Aboba, Principal Architect, Microsoft Skype
Updated January 31, 2017 10:07 am
Authored by Michelle Bergeron [MSFT]
UsbCApiCmd is a simple tool that you can use for basic validation of your Type-C implementation on Windows.
UsbCApiCmd is applicable only to USB Type-C Connectors which use Microsoft’s USB Type-C Stack, the USB Connector Manager. UcmCx.sys needs to be loaded and running on the system. The tool is not applicable to systems which use other methods to manage the USB Type-C ports.
UsbCApiCmd is included in the MUTT Software Package.
Open a command prompt or PowerShell window on the system with the USB Type-C connector(s) you wish to validate. Run UsbCApiCmd.exe.
For as long as the program is running, the console will print out information about the USB Type-C connectors on the system that are registered with the USB Connector Manager (UcmCx). It will also print out information about detected Attach/Detach events on a connector. You can use it to validate that the Type-C software is seeing the events that you expect it to.
To stop execution, you may press Ctrl+C.
The below example is from a system using UCSI to manage its connectors with Windows. It has not detected any attach or detach events since the program started running.
Planned Maintenance: Saturday 04 February 2017 17:00 UTC – Saturday 04 February 2017 21:00 UTC
We will be performing planned maintenance on a VS Team Services instance hosted in Azure’s South Central US region. This maintenance is needed to deploy platform updates which will improve the quality of our service.
During this maintenance, we do not anticipate any user impact. Customers connecting to their Git repos using SSH authentication may receive an expected warning that IP address has been changed. Should you experience any errors during this time, please retry your operation and it should succeed.
We appreciate your use of our service and we are committed to delivering a great experience that improves constantly.
Sincerely,
Sri Harsha
Chip-ing AG is a startup based in Switzerland that produces a smart, connected golf ball designed to help golfers locate it if it gets lost as well as coach them to improve their game.
The ball can easily be located on the course through a mobile app, saving precious time and frustration for both the player as well as other players on the course, waiting for their turn to play. Additionally, the smart ball contains sensors such as accelerometers that help analyze the play style, offering endless possibilities such as virtual game coaching, automatic scoring, and much more.
The solution consists of a mobile iOS and Android app acquiring data from the Bluetooth Low Energy (BLE) ball during play. It combines this with data from the device sensors and sends it to Azure IoT Hub. Stream Analytics then splits the data and provides the first insight in real time and moves the data to a persistent storage (SQL, DocumentDB or Table service).
Power BI provides a quick visualization of the data, giving Chip-ing key insights into how players are using their product. Chip-ing plans to implement Machine Learning with the goal of improving the location algorithm by correlating data such as signal strength, device heading, and GPS position, thereby making the ball retrieval process as efficient as possible. In a second step, the data analysis should also give more insights into the player’s golfing technique, possibly improving the player’s game.
During a workshop between Microsoft and Chip-ing, we were able to create a first working proof of concept within a few days. The combination of IoT Hub, Stream Analytics, SQL, and Power BI proved to be a very simple yet powerful way to tackle our problem with rapid prototyping. Having a platform that allows you to easily scale to thousands of customers using tens of thousands of golf balls becomes feasible even for a small startup. Azure provides complete management and diagnostic tools that are easy enough to use, allowing even a small team to run a highly scalable production environment.
Have a look at the online Case Study that explains the technologies used in greater detail as well as includes many code samples.
Chip-ing AG est une start-up basée en Suisse qui fabrique des balles de golf intelligentes et connectées afin d’aider les golfeurs à les retrouver s’ils les perdent et de les coacher afin qu’ils se perfectionnent.
Ils peuvent les retrouver grâce à une application mobile, économisant du temps précieux pour les utilisateurs ainsi que pour les autres joueurs, qui attendent leurs tours pour jouer. De plus, les balles intelligentes contiennent des capteurs permettant d’analyser le style de jeu et procurant des possibilités infinies dans les domaines du coaching virtuel, du suivi du score, etc.
Pendant la partie, la solution comprend une application mobile iOS ou Android qui collecte des données sur la balle alimentée par la technologie Bluetooth Low Energy (BLE). Elle les corrèle à celles des capteurs et les envoie à Azure IoT Hub. Stream Analytics sépare les données et fournit les premières analyses en temps réel, puis transfère les données dans un emplacement de stockage persistant (SQL, DocumentDB, ou Table service).
Power BI permet de visualiser les données et à Chip-ing de comprendre comment les joueurs utilisent leur produit. Chip-ing prévoit d’implémenter un apprentissage automatique afin d’optimiser l’algorithme de localisation en corrélant des données tel que la puissance du signal, l’orientation géographique ou encore la position GPS. L’analyse des données fournira plus d’informations sur la technique des joueurs.
Lors d’un workshop avec Microsoft et Chip-ing, nous avons pu réaliser la première validation technique fonctionnelle. Il s’est avéré que l’association d’IoT Hub, de Stream Analytics, de SQL et de Power BI était un moyen simple de résoudre notre problème en réalisant rapidement un prototype. Ainsi, même une petite start-up peut disposer d’une plate-forme évolutive pouvant gérer des milliers de clients qui utilisent des dizaines de milliers de balles de golf. Azure dispose d’outils d’administration et de diagnostic complets et simples à utiliser, permettant à une équipe de gérer un environnement de production très évolutif.
Jetez un coup d’œil à l’étude de cas en ligne qui présente les technologies utilisées et inclut également des exemples de codes.
Chip-ing AG ist ein in der Schweiz ansässiges Startup-Unternehmen, das einen intelligenten vernetzten Golfball herstellt. Er soll Golfspielern helfen, den Ball wiederzufinden, wenn er verschwunden ist, und ihnen dabei helfen, ihr Spiel zu verbessern.
Der Ball ist mittels einer Mobil-App problemlos auf dem Golfplatz zu lokalisieren, was dem Spieler selbst ebenso wie anderen Spielern auf dem Platz, die darauf warten, an die Reihe zu kommen, wertvolle Zeit und Frustration erspart. Ausserdem verfügt der Golfball über Sensoren wie Beschleunigungsmesser, die helfen, den Stil des Spielers zu analysieren, und zahllose Möglichkeiten wie virtuelles Coaching oder automatische Bewertung etc. bieten.
Die Lösung besteht aus einer Mobil-App für iOS und Android, die während des Spiels Daten des «Bluetooth Low Energy (BLE)»-Balls erfasst. Sie kombiniert diese Daten mit den Sensordaten des Geräts und sendet sie an den Azure-IoT-Hub. Stream Analytics teilt diese Daten auf, liefert einen ersten Einblick in Echtzeit und legt die Daten in einem persistenten Speicher (SQL-, DocumentDB- oder Table-Dienst) ab.
Power BI bietet eine schnelle Visualisierung der Daten und liefert Chip-ing wichtige Erkenntnisse darüber, wie die Spieler ihr Produkt nutzen. Chip-ing plant, Maschinenlernen zu implementieren, mit dem Ziel, den Lokalisierungsalgorithmus zu verbessern, indem Daten wie Signalstärke, Richtung des Geräts und GPS-Position korreliert werden, um das Wiederfinden des Balls so effizient wie möglich zu gestalten. In einem zweiten Schritt soll die Datenanalyse auch mehr Einblick in die Golftechnik des Spielers ermöglichen, um gegebenenfalls sein Spiel zu verbessern.
Während eines Workshops von Microsoft und Chip-ing konnten wir innerhalb weniger Tage einen ersten funktionierenden Proof of Concept erstellen. Die Kombination aus IoT-Hub, Stream Analytics, SQL und Power BI hat sich als sehr simple und doch leistungsfähige Möglichkeit erwiesen, unser Problem mit dem Rapid Prototyping zu lösen. Über eine Plattform zu verfügen, die auf Tausende Kunden mit Zehntausenden Golfbällen skaliert, wird sogar für ein kleines Startup-Unternehmen möglich. Azure bietet umfassende Verwaltungs- und Diagnosewerkzeuge, die bedienungsfreundlich sind und es auch einem kleinen Team ermöglichen, eine hochskalierbare Produktionsumgebung zu betreiben.
Sehen Sie sich die Online-Fallstudie an, die die eingesetzten Technologien detaillierter erläutert und zahlreiche Codebeispiele umfasst.
Digitale Technologien sind ein wichtiger Schlüssel für die Chance auf Bildung, Wissen und beruflichen Erfolg in unserer Gesellschaft. Aus diesem Grund ist Microsoft auch in diesem Jahr gleich auf zwei Ständen der didacta 2017 – Deutschlands wichtigster Bildungsmesse – vertreten. Im Digitalen Lernlabor können Besucher vom 14. bis 18. Februar erleben, wie digitale Bildung in der Praxis aussieht: Mit der Initiative Code your Life können Lehrer, Bildungspraktiker, Schüler und Eltern direkt in die Welt des Programmierens eintauchen. Auf dem Gemeinschaftsstand mit dem Bündnis für Bildung (BfB) präsentiert Microsoft sein breites Bildungsportfolio und bietet im „Forum Didacta Digital – Besser lernen in der Praxis“ ein abwechslungsreiches Vortragsprogramm mit vielen Anwendungsbeispielen aus dem Bildungsalltag.
„Wir wollen möglichst vielen Schülern und Lehrern Zugang zu aktuellen Informationstechnologien bieten und setzen uns dafür ein, die Kompetenz im Umgang mit digitalen Medien zu stärken. Auf der didacta zeigen wir, wie der Einsatz digitaler Technologien im Unterricht lebendigere Lernerfahrungen und nachhaltigere Lernerfolge ermöglicht“, sagt Renate Radon, Mitglied der Geschäftsleitung von Microsoft Deutschland und dort zuständig für den Public Sector.
Coding – Making – Rocking: Angebote im Digitalen Lernlabor
Im Digitalen Lernlabor (Halle 4, Stand B80), können Besucher live erleben, wie sich Kinder und Jugendliche unterschiedlicher Altersgruppen für das Programmieren begeistern. Sie tauchen in drei inhaltlich fokussierte Lernszenarien ein und erfahren hautnah das kreative Potential des Programmierens. Im Bereich Coding geht es um Programmieren in seiner Grundform. Der Bereich Making widmet sich Experimenten mit Mikrocontrollern und Sensoren und im Bereich Rocking lernen Kinder, wie sich mithilfe von Codes elektronische Beats erstellen lassen, die zu einzelnen Sounds oder ganzen Melodien werden. Dabei kommen alle aktuellen Minicomputer, ob Calliope, micro:bit, Arduino oder Raspberry Pi sowie der Zeichenroboter Mirobot zum Einsatz.
Lehrkräfte erfahren am Stand alles über die Initiative, neue Arbeitsmaterialien und Tools zum Programmieren und können dabei von und gemeinsam mit den Kindern lernen. Die Experten von Code your Life stehen während der Messe für Fachkräfte aus Schule, Kinder- und Jugendarbeit sowie für Eltern zu Fragen der digitalen Bildung bereit. Ein Begleitprogramm zum digitalen Lernen mit Kurzvorträgen, Coding-Shows und eine tägliche Tablet-Verlosung runden das Angebot ab.
#besserlernen in der Praxis
Darüber hinaus ist Microsoft am Stand des Bündnisses für Bildung (Halle 4, Stand A30.1) vertreten und präsentiert dort sein breites Bildungsportfolio. Dieses reicht von aktuellen Surface-Modellen bis zu Office 365 für Bildungseinrichtungen, das ab sofort auch in der Microsoft Cloud Deutschland verfügbar ist. Auf dem „Forum Didacta Digital – Besser lernen in der Praxis“ finden täglich Vorträge und Workshops statt, in denen Akteure und Praktiker aus dem Bildungsbereich über ihre Erfahrungen berichten und konkrete Lösungen für den digitalen Unterricht vorstellen.
Renate Radon, Mitglied der Geschäftsleitung von Microsoft Deutschland, diskutiert am Messefreitag im Rahmen des „Forums didacta aktuell“ (Halle 5, Stand D32) zum Thema „Wieviel Wirtschaft verträgt Schule?“ und steht darüber hinaus für Gespräche während der Messe zur Verfügung.
didacta-Highlights auf einen Blick:
Dienstag, 14. Februar: Offener Makerspace mit Schülern und Lehrkräften;
Beratung für Lehrkräfte: Die Schule als Makerspace (Digitales Lernlabor: Halle 4, Stand B80)
Dienstag, 14. Februar: Produktiver Einsatz von Tablets und Office 365 im Unterricht; Praxisbeispiele; Referent: Dieter Feuer, Ludwig-Erhard-Schule, Sigmaringen; 11 bis 11.30 Uhr (Forum Didacta Digital – Besser lernen in der Praxis, Halle 4; Stand A30.1)
Mittwoch, 15. Februar: Schwerpunkt Berufsschulen – Coding in der beruflichen Ausbildung; Beratung für Lehrkräfte: Coding in der Berufsschule (Digitales Lernlabor: Halle 4, Stand B80)
Mittwoch, 15. Februar: Digitale Bildung auch für die technikferne Lehrkraft – wie bekomme ich alle Kollegen ins Boot; Vortrag von Jan Weiss, Lehrer und Netzwerkberater aus Filderstadt; 10.30 bis 11 Uhr; (Forum Didacta Digital – Besser lernen in der Praxis, Halle 4; Stand A30.1)
Donnerstag, 16. Februar: Heiter bis wolkig – Training aus der Cloud; Vortrag von Sven Koritnik und Bassem Serhan, Ministerialreferenten, Ministerium für Kultus, Jugend und Sport Baden-Württemberg; 11 bis 11.30 Uhr (Forum Didacta Digital – Besser lernen in der Praxis, Halle 4; Stand A30.1)
Donnerstag, 16. Februar: Schlaumäuse-Initiative: Presse-Termin in Kooperation mit dem Deutschen Städte- und Gemeindebund und Übergabe eines Schlaumäuse-Starterpakets an eine Bildungseinrichtung. (Halle 4; BfB, Stand A30.1)
Freitag, 17. Februar: Schwerpunkt Coding & Minecraft – Der Minecraft-Abend im Lernlabor; Beratung für Lehrkräfte: Coding in der Sekundarstufe / Mit Minecraft lernen (Digitales Lernlabor: Halle 4, Stand B80)
Samstag, 18. Februar: Familientag – Eltern programmieren mit ihren Kindern; Elternberatung: Warum sollte ich mit meinem Kind programmieren lernen? (Digitales Lernlabor: Halle 4, Stand B80)
Mehr Informationen zur Initiative Code your Life finden Sie hier: www.code-your-life.org/.
Weitere Informationen zu Veranstaltungen und Vorträgen auf dem Gemeinschaftsstand des Bündnisses für Bildung finden Sie hier.
Weiterführende Informationen zu den Bildungsangeboten von Microsoft finden Sie unter: www.microsoft.de/besserlernen.
Auf Twitter können Sie dem Hashtag #besserlernen folgen und so an der Debatte rund um die Digitalisierung des Bildungswesens teilhaben.
In my previous post, I described how to leverage your R skills using Microsoft technologies for ordinary business users (marketers, business analysts, etc) with Power BI.
In this post, I describe what is the benefits of Microsoft R technologies for professional developers (programmers, data scientists, etc) with a few lines of code.
R is most popular statistical programming language, but it’s having some concerns for enterprise use. The biggest one is the lack of parallelism.
Microsoft R Open (MRO) is renamed from the famous Revolution R Open (RRO). By using MRO you can take advantage of multithreading and high performance, although the functions of MRO is still compatible for the basic functions of open source R like CRAN.
Note : You can also use other choice (snow, etc) for parallel computing in R.
Please see the following official document about the benchmark.
The Benefits of Multithreaded Performance with Microsoft R Open
https://mran.microsoft.com/documents/rro/multithread/
For example, this document says that the matrix manipulation is many times faster than the open source R.
Let’s see the following simple example. The A is 10000 x 5000 matrix which elements is the repeated values of 1, 2, 3, 4, 5, 1, 2, 3, 4, 5, 1, 2, … The B is the cross-product matrix by A. Here we measure this cros-product operation using system.time().
A <- matrix (1:5,10000,5000)
system.time (B <- crossprod(A))The following is the results.
Here I’m using Lenovo X1 Carbon (with Intel Core vPro i7 and Integrated Intel HD Graphics), and MRO is over 8 times faster than the open source R.
R 3.3.2 [64 bit] (CRAN)
Microsoft R 3.3.2 [64 bit] (MRO)
Note : If you’re using RStudio and installing both open source R and MRO, you can change the R runtime for RStudio from [Tools] – [Global Options] menu.
By using Microsoft R Server (formerly, Revolution R Enterprise), you can also distribute and scale the computing of R across the multiple computers.
The R Server can be run on Windows (SQL Server), Linux, Teradata, and Hadoop (Spark) clusters.
You can also use R Server as one of the workload on Spark (see the following illustrated), and here we use the Spark cluster in this post. Using R Server on Spark, you can distribute the R algorithm on Spark cluster.
Note : For Windows, R Server is licensed under SQL Server. You can easily get the standalone R server (with SQL Server 2016 Developer edition) by using the virtual machine (called “Data Science Virtual Machine”) in Microsoft Azure.
Note : Spark MLlib is also the machine learning component to get the power of computing on Spark, but Python, Java, and Scala are the mainly used programming languages. (Currently almost functions are not supported in R.)
Note : You can also use SparkR. But you must remember that the naked SparkR is currently just the data transformation for R computing (i.e, not mature). R Server is calling SparkR on the basis.
Here I skip how to setup your R Server on Spark, but the easiest way is to use Azure Hadoop clusters (called “HDInsight”). You can setup your own experiment environment by just a few steps as follows.
Note : Here I used Azure Data Lake store (not Azure Storage Blobs) for the primary storage on Hadoop clusters. (And I setup the service principal and its access permissions for this Data Lake account itself.)
For more details, please refer “Create an HDInsight cluster with Data Lake Store using Azure Portal“.
Using RStudio Server on the edge node on Spark cluster, you can use RStudio on the web browser using SSH tunnel. (See the following screenshot.)
Here I prepared the source data (Japanese stocks daily reports) over 35,000,000 records (over 1 GB).
When I start to analyze using R on my local computer, the script fails because of the allocation error or timeout. In such a case, you can solves this problem using R Server on Spark cluster.
Now here’s the R script which I run on R Server.
##### The format of source data
##### (company-code, year, month, day, week, open-price, difference)
#
#3076,2017,1,30,Monday,2189,25
#3076,2017,1,27,Friday,2189,-1
#3076,2017,1,26,Thursday,2215,-29
#...
#...
#####
# Set Spark clusters context
spark <- RxSpark(
consoleOutput = TRUE,
extraSparkConfig = "--conf spark.speculation=true",
nameNode = "adl://jpstockdata.azuredatalakestore.net",
port = 0,
idleTimeout = 90000
)
rxSetComputeContext(spark);
# Import data
fs <- RxHdfsFileSystem(
hostName = "adl://jpstockdata.azuredatalakestore.net",
port = 0)
colInfo <- list(
list(index = 1, newName="Code", type="character"),
list(index = 2, newName="Year", type="integer"),
list(index = 3, newName="Month", type="integer"),
list(index = 4, newName="Day", type="integer"),
list(index = 5, newName="DayOfWeek", type="factor",
levels=c("Monday", "Tuesday", "Wednesday", "Thursday", "Friday")),
list(index = 6, newName="Open", type="integer"),
list(index = 7, newName="Diff", type="integer")
)
orgData <- RxTextData(
fileSystem = fs,
file = "/history/testCsv.txt",
colInfo = colInfo,
delimiter = ",",
firstRowIsColNames = FALSE
)
# execute : rxLinMod (lm)
system.time(lmData <- rxDataStep(
inData = orgData,
transforms = list(DiffRate = (Diff / Open) * 100),
maxRowsByCols = 300000000))
system.time(lmObj <- rxLinMod(
DiffRate ~ DayOfWeek,
data = lmData,
cube = TRUE))
# Here's just ploting the means of DiffRate for each DayOfWeek
lmResult <- rxResultsDF(lmObj)
rxLinePlot(DiffRate ~ DayOfWeek, data = lmResult)
# execute : rxCrossTabs (xtabs)
system.time(ctData <- rxDataStep(
inData = orgData,
transforms = list(Close = Open + Diff),
maxRowsByCols = 300000000))
system.time(ctObj <- rxCrossTabs(
formula = Close ~ F(Year):F(Month),
data = ctData,
means = TRUE
))
print(ctObj)Here we uploaded the source data on Azure Data Lake store which is the same storage as the primary storage of Hadoop cluster. “adl://...” means the uri of Azure Data Lake store account.
The above functions which is prefixed by “rx” are called ScaleR functions. These functions are provided for distributing and scaling, and each ScaleR function is the scaling one of the corresponding basic R functions. For example, RxTextData is corresponding to read.table or read.csv, rxLinMod is corresponding to lm (linear regression model), and rxCrossTabs is xtabs (cross-tabulation).
You can use these R functions for leveraging the computing power of Hadoop clusters. (See the following reference document for details.)
Microsoft R – RevoScaleR Functions for Hadoop
https://msdn.microsoft.com/en-us/microsoft-r/scaler/scaler-hadoop-functions
Microsoft R – Comparison of Base R and ScaleR Functions
https://msdn.microsoft.com/en-us/microsoft-r/scaler/compare-base-r-scaler-functions
Note : For more details (descriptions, arguments, etc) about each ScaleR functions, please type “?{function name}” (ex. “?rxLinePlot“) in R Console.
The following illustrates the topology of Spark clusters. The process of R Server resides in the edge node and worker nodes.
The edge node is having the role of the development front-end, and you can interact with R Server through this node. (Currently, R Server on HDInsight is the only cluster which provides the edge node by default.)
For example, RStudio Server is installed on this edge node, and when you run your R scripts through RStudio on the web browser, this node starts all computations distributed to worker nodes. If the computation cannot be distributed for some reason (both intentionally and accidentally), this task will be run on this local edge node.
While the script is running, please see YARN on resource manager (rm) in Hadoop. You could find the running SparkR application on the scheduler. (See the following screenshot.)
When you monitor the nodes in resource manager UI, you could find that all nodes are used for computation.
The ScaleR functions are also provided in Microsoft R Client (on top of Microsoft R Open), which can run on the standalone computer. (You don’t need extra servers.)
Using Microsoft R Client, you can send completed R commands to the remote R Server for execution. Or, you can learn and test these ScaleR functions on the local computer with Microsoft R Client, and you can migrate to the distributed clusters.
I think it’s better idea to use Microsoft R Client in the development time, because there’re so many overheads (fee, provisioning, etc) to use Hadoop clusters when you’re in the development.
You can take a lot of advantages of the robust computing platform with Microsoft R technologies !
皆さん、こんにちは。テクニカルエバンジェリストの戸倉彩です。
マイクロソフト本社のMicrosoft Imagine(読み方:マイクロソフト イマジン)チームが公開しているGitHubに、今注目されている人工知能が体験できる「Cognitive Services(読み方:コグニティブサービス)」の一部となる自然言語解析サービス『LUIS(Language Understanding Intelligent Services)』のワークショップやハンズオン等に参考になるコンテンツが新たに加わりましたので、ご紹介させていただきます。
■LUIS Hands on Labo (英語)
https://github.com/MSFTImagine/computerscience/blob/master/Workshop/14.%20LUIS/LUIS%20HOL.md
■このHands on Laboで学べること
・LUISアプリケーションの作成 (Create a LUIS application)
・インテントとエンティティを定義する (Define intents and entities)
・事前に作成されたエンティティおよびフレーズリストを利用する (Use pre-built entities and phrase lists)
・LUISモデルをトレーニングし、それをHTTPエンドポイントに公開する (Train a LUIS model and publish it to an HTTP endpoint)
・LUISモデルを使用するbotを作成する (Create a bot that uses a LUIS model)
■前提条件
この実習を行うためには、下記が必要です。
・Microsoftアカウント ※Microsoftアカウントをお持ちでない方はコチラを参考にしてアカウント取得してください。
・Visual Studio Code 1.7.0以降 ※OS環境はWindows、MacOS、Linuxに対応
・WEBブラウザ(IE 10以降、またはGoogle Ghrome)
・Node.js
■所要時間の目安
60分。
実習は、Exercise 1～６の流れに沿って手を動かしていきます。
すべて英語で書かれていますが、スクリーンショットがたくさん使われていますので、分かりやすいと思います。
ワタシのWindows 10環境で、Visual Studio Code 1.8.1を利用して最後まで試してみたところ、16番目の「LUIS_APP_ID」と19番目の「LUIS_SUBSCRIPTION_ID」の手順に気をつけることで、スムーズに進めることができました。完成した後は、日本語でオリジナルの対話ができるようにアレンジしてみると面白いと思います。
Have a nice LUIS♪
※Twitterで最新情報発信中 @ayatokura
The README.md file in the repo for the DirectX HLSL Compiler provides the steps on how to build the project. I thought it would be interesting to share some of our journey about how we landed where we currently are, and what we think moving forward looks like.
The obvious starting point for this project was the existing LLVM/Clang build system. We especially liked the CMake-based approach and its integration with Microsoft Visual Studio, and so we based the build experience around that. The other dependencies were then picked as needed:
This is a fairly long list! However we believe that we have provided straightforward instructions on how to get everything lined up, and we’ve added some amount of setup verification to the “HLSL Console Tools” set of scripts – more on those in a future post.
In the future, we expect we’ll be updating to newer versions of the tools we’ve selected, and we’ll try to simplify installation and setup.
Enjoy!
Some network environments are locked down via a Firewall and allow only whitelisted IP addresses inbound to their internal network. Common questions are, “what is my Azure Web App, Azure Mobile App (insert your type of Azure App Service here) outbound IP address”? What IP addresses do I need to whitelist for Azure? Even if you determine it through a network trace or firewall log, you will notice it can change. This is because of the internal structure of Azure. The outbound address could be any address within a certain range. However, this range is not static.
We provide a list of the Azure Data Center IP ranges. You can download this XML file and find the Data Center that your Azure Web App is in and then whitelist those ranges of IPs. This list is update weekly (mostly with additions). Then you can automate or manually update your firewall with this list to ensure your app is able to call into your network through your firewall.
Here is an example:
The app I want to find the range of IPs for is called problemwebapp. I look this up in the Azure Portal to see what Data Center is in (in the Overview tab) and I see it is in ‘East US’:
Next I go to this link and download the XML File with the Azure Data Center IP listing: Microsoft Azure Datacenter IP Ranges
And search for the location. I searched on ‘us’, then ‘east’ and was able to find this (yes it is a bit cryptic but it will not change):
At this point you have the information for that Data Center and can add these IP ranges to your Firewall. Depending on how your firewall is managed, you may be able to automate this (talk to your firewall provider).
What does this list include:
Will I have time to update my firewall before a change happens:
How often can this outbound list change:
Is this only Web Apps or ALL IPs in the data center:
Per Microsoft Azure Datacenter IP Ranges, all of these questions above are answered in the Details section:
https://www.microsoft.com/en-us/download/details.aspx?id=41653 “This file contains the Compute IP address ranges (including SQL ranges) used by the Microsoft Azure Datacenters. A new xml file will be uploaded every Wednesday (Pacific Time) with the new planned IP address ranges. New IP address ranges will be effective on the following Monday (Pacific Time). Please download the new xml file and perform the necessary changes on your site before Monday.“
Is there a way to narrow this list to just Azure Web Apps: No
We are working hard to resolve this issue and apologize for any inconvenience.
-Arun
Note: The new feature(s) covered in this post are available to Windows Insiders running build 15007 or newer. If you are using the UWP Toolkit Notifications library, the feature(s) can be found in the beta version here.
New in Creators Update: In the most recent Windows 10 Creators Update, we added several exciting improvements and new features to toast notifications and Action Center. One of them is allowing developers to show an animated progress indicator inside their toast notifications to show the progress of an ongoing operation, such as file transfer, progress of completing a task, progress of a work-out session, etc.
A progress bar inside a toast can be either “determinate” to show a specific value, or “indeterminate” which can be used when the app does not know the length of the operation. For a “determinate” progress indicator in a toast notification, an app needs to keep on updating the status and value of it using notification data binding, another new feature that will be demonstrated later in the post.
Note: Mobile does not support the progress indicator as part of the Creators Update, but support will come in a future Windows release. Notification data binding as a stand-alone feature will be supported in Mobile in Creators Update so apps can use that to update text content inside toast notifications, which will be covered later in this blog post.
As you can see in the picture below, here is what a progress indicator in a Windows toast notification looks like：
In this notification, you can see that a progress indicator contains several visual elements, that are either required or optional.
| Visual element | Type and value | Description |
|---|---|---|
| title | string | Title is an optional attribute – it is a custom string that is displayed above the animated progress indicator. It can be used to explain what the progress is indicating; |
| status | string | Status is a requried attribute – it is a custom string that app can provide to be displayed below the animated progress indicator. It can be used to explain the status of the progress, such as downloading, uploading, paused, error, etc. |
| value | A double between 0 and 1 or string “indeterminate” | Value is a required attribute.
You can provide a double between 0 and 1 (inclusive) that indicates the actual progress – this will directly result in the progress bar being draw with x% full; Or, you can use the string value “indeterminate”, which will result in the “indeterminate” progress bar animation. Any value other than the above would be considered invalid and will cause undesired behaviors. |
| valueStringOverride | string | ValueStringOverride is an optional attribute – it is a custom string value that apps can use to override the default automatic textual representation of their progress value, which is displayed at the bottom right corner below the progress indicator. Apps can use this to clarify the semantic meaning of the progress value (for example, when you are transferring 10 files (5 MB) in total, when you provide a value of 0.2, you can further clarify that whether this means 4/10 files or 1/5 MB transffered using the valueStringOverride property);
If not provided, a default string like 20% will automatically be displayed. |
Here is how to specify the above progress indicator inside XML:
<toast>
<visual>
<binding template="ToastGeneric">
<text>Getting closer to this week's exercise goal!</text>
<progress title="Ab Ripper" value="0.6" valueStringOverride="3/5 sessions" status="On track"/>
</binding>
</visual>
</toast>
However, you’ll need to dynamically update the values of the progress indicator for it to actually be “live”. This can be done by updating a toast notification with data-binding.
There are two parts to updating a toast notification – sending an initial toast notification with data bound fields, and updating those fields with new values when there is any update.
Note: In the upcoming Creators Update, all attributes of the <progress> element, and the text contents of top-level <text> elements support data binding inside a toast notification payload.
First, the app needs to construct the initial toast notification payload with data binding fields in it, and use the Data property to provide the corresponding initial data values.
using Windows.UI.Notifications;
public void SendUpdatableToastWithProgress()
{
// Define a tag value and a group value to uniquely identify a notification, in order to target it to apply the update later;
string toastTag = "pikachu_0132";
string toastGroup = "egg_fertilization";
// Construct the toast payload with updatable data fields inside;
string toastXmlString =
@"
Keep walking! A cute Pikachu will be your reward!
";
XmlDocument toastXml = new XmlDocument();
toastXml.LoadXml(toastXmlString);
// Generate the toast notification;
var toast = new ToastNotification(toastXml);
// Assign the tag and group properties;
toast.Tag = toastTag;
toast.Group = toastGroup;
// Define NotificationData property and add it to the toast notification to bind the initial data;
// Data.Values are assigned with string values;
toast.Data = new NotificationData();
toast.Data.Values["progressValue"] = "0.01";
toast.Data.Values["progressString"] = "0.1/10 miles";
toast.Data.Values["progressStatus"] = "Just getting started";
// Provide sequence number to prevent updating out-of-order or assign it with value 0 to indicate "always update";
toast.Data.SequenceNumber = 1;
// Show the toast notification to the user;
ToastNotificationManager.CreateToastNotifier().Show(toast);
}
Then, when there is new content or update you want to show to the user, simply provide the new data to update the toast without re-constructing the entire payload by calling the Update() method.
using Windows.UI.Notifications;
public void UpdateProgress()
{
// Construct a NotificationData object;
string toastTag = "LongRunningOperationXYZ";
string toastGroup = "LongRunningOperations";
// Create NotificationData with new values;
// Make sure that sequence number is incremented since last update, or assign with value 0 for updating regardless of order;
var data = new NotificationData { SequenceNumber = 2 };
data.Values["progressValue"] = "0.5";
data.Values["progressStatus"] = "running";
// Updating a previously sent toast with tag, group, and new data;
NotificationUpdateResult updateResult = ToastNotificationManager.CreateToastNotifier().Update(data, tag, group);
}
Here are some more details about the newly added class and properties:
NotificationData Class
NotificationData contains the data needed to update a notification.
| Constructor |
| NotificationData (IIterable<IKeyValuePair<HSTRING, HSTRING>>, ULONG) |
| NotificationData (IIterable<IKeyValuePair<HSTRING, HSTRING>>) |
| NotificationData() |
| Property | Description |
| Values IMap<string, string> | The actual values of the data in key-value pairs; |
| SequenceNumber ULONG | The SequenceNumber of the update to prevent false update caused by race conditions Note that the value 0 means “always update.” |
Both ToastNotification and ScheduledToastNotification have added new Data property.
| Property and Type | Description |
| ToastNotification.Data
NotificationData |
Define the data property to provide initial values of data when a toast notification is using data-binding and is expected to be updated afterwards. |
| Property and Type | Description |
| ToastNotification.Data
NotificationData |
Define the data property to provide initial values of data when a toast notification is using data-binding and is expected to be updated afterwards.Define the data property to provide initial values of a scheduled toast notification when this notification is using data-binding and is expected to be updated afterwards. |
ToastNotifier.Update()
| Method | Syntax | Description |
| ToastNotifier.Update(Tag, Group, NotificationData) | public enum Update( string tag string group NotificationData data ) |
Call this method to update on an existing toast notification based on the tag and group properties, when the original toast has tag and group values provided. It returns an enum which indicates the result of the update. |
| ToastNotifier.Update(Tag, NotificationData) | public enum Update( string tag NotificationData data ) |
Call this method to update on an existing toast notification based on the tag property, when the original toast has tag value provided. It returns an enum which indicates the result of the update. |
Here are details on NotificationUpdateResult enum:
| Member | Value | Description |
| Succeeded | 0 | Notification is successfully updated |
| Failed | 1 | Notification failed to be updated for unknown reason |
| NotificationNotFound | 2 | Notification failed to be updated because the original notification was missing |
Note: If the time interval between your initial notification delivery and your notification update is long, there is a good chance that you will get NotificationNotFound result because the user has already dismissed the notification. This could potentially mean that he/she is not interested in further updates on this content either. We recommend you to think carefully based on your specific scenario before popping another toast for the same content.
Some developers may already know that you can replace a previously sent notification by re-sending a new toast with the same Tag and Group values. So what exactly is the difference between updating and replacing a notification, and how to decide on when to use one or the other?
New in Creators Update: You can finally programmatically pin your own app’s primary tile to Start, just like secondary tiles! And you can check if it’s currently pinned. Note that you must target Insider SDK 15003 and be running Insider build 15025 or newer to use the primary tile API.
At its core, the API lets you…
You put a lot of effort into designing a great experience for your app’s primary tile, and now you have the opportunity to ask the user to pin it to Start. But before we dive into the code, here are some things to keep in mind as you’re designing your experience:
If your app supports older versions of Windows 10, you need to check whether these primary tile API’s are available. You do this by using ApiInformation. If the primary tile API’s aren’t available, avoid executing any calls to the API’s.
if (ApiInformation.IsTypePresent("Windows.UI.StartScreen.StartScreenManager"))
{
// Primary tile API supported!
}
else
{
// Older version of Windows, no primary tile API
}
Depending on the current Start menu, and your type of app, pinning your app to the current Start screen might not be supported. Only Desktop and Mobile support pinning the primary tile to Start. Therefore, before showing any pin UI or executing any pin code, you first need to check if your app is even supported for the current Start screen. If it’s not supported, don’t prompt the user to pin the tile.
// Get your own app list entry // (which is always the first app list entry assuming you are not a multi-app package) AppListEntry entry = (await Package.Current.GetAppListEntriesAsync())[0]; // Check if Start supports your app bool isSupported = StartScreenManager.GetDefault().SupportsAppListEntry(entry);
To find out if your primary tile is currently pinned to Start, use the ContainsAppListEntryAsync method.
// Get your own app list entry AppListEntry entry = (await Package.Current.GetAppListEntriesAsync())[0]; // Check if your app is currently pinned bool isPinned = await StartScreenManager.GetDefault().ContainsAppListEntryAsync(entry);
If your primary tile isn’t currently pinned, and your tile is supported by Start, you might want to show a tip to users that they can pin your primary tile.
Note: You must call this API from a UI thread while your app is in the foreground, and you should only call this API after the user has intentionally requested the primary tile be pinned (for example, after the user clicked yes to your tip about pinning the tile).
If the user clicks your button to pin the primary tile, you would then call the RequestAddAppListEntryAsync method to request that your tile be pinned to Start. This will display a dialog asking the user to confirm that they want your tile pinned to Start.
This will return a boolean representing whether your tile is now pinned to Start. If your tile was already pinned, this will immediately return true without showing the dialog to the user. If the user clicks no on the dialog, or pinning your tile to Start isn’t supported, this will return false. Otherwise, the user clicked yes and the tile was pinned, and the API will return true.
// Get your own app list entry AppListEntry entry = (await Package.Current.GetAppListEntriesAsync())[0]; // And pin it to Start bool isPinned = await StartScreenManager.GetDefault().RequestAddAppListEntryAsync(entry);
The new Business process modeler (BPM) tool has been released and is currently available in preview mode. If you are interested in previewing the new functionality, contact Microsoft and you will be given a preview code.
Complete the following steps to enable the preview functionality.
Use the following tasks to create a BPM library or make changes to an existing library.
To add a new process, select an existing process in your library, then click Add process on the toolbar. You can add a process as a child or sibling of the selected process node. This enables you to create a semantic hierarchy of business processes.
Complete the following steps to edit the properties of a business process.
You can move or assign a business process node to another parent node in the BPM hierarchy.
To delete a business process, select the process that you want to delete, and then on the toolbar, select Delete.
You can browse a BPM library that is a Global or Corporate library. However, to edit and work with a BPM library, it needs to be part of your LCS project. Libraries distributed by Microsoft appear under Global libraries, while libraries that are published by your organization appear under Corporate libraries.
You can search for words or phrases in your BPM library. The search functionality searches through business process names and descriptions. To search for a word, enter the word in the search bar and press Enter. To search for a phrase, use double quotes around the search phrase. For example, enter technology (word) or “information technology” (phrase) in the search bar.
You can also search for AOT elements that are part of the Dynamics 365 for Operations Task recordings in your library. These are typically form names or menu items names. When you search for an AOT element, prefix it with the $ sign. For example, $CustTable.
You can associate an activity diagram with a business process. Activity diagrams are used to describe how a business process or task is completed in a proposed software solution.
There are 3 types of activity diagrams:
In addition to activity diagrams, you can describe a business process using detailed process steps.
The Diagrams column in your BPM library indicates when a particular business process is associated with an activity diagram. The number indicates the number of child processes that include diagrams and the icon next to the number indicates whether the current node/process is associated with a diagram. These indicators do not apply to Visio diagrams.
To view or edit an activity diagram, navigate to the business process, and then in the Overview pane, select Diagrams. This will open the Flowchart page.
You can create a Task recording in your Dynamics 365 for Operations environment and save it directly to LCS. This will associate a Task recording with a business process in a BPM library. For more information, see Connecting the help system and Create documentation or training using task recordings.
The task recorder tool in Dynamics 365 for Operations allows you to create a distributable recording file (.axtr). You can associate a business process in BPM with a Task recording by manually uploading a recording (.axtr) file. In BPM, select the desired business process then select Upload.
BPM automatically generates an activity diagram and detailed process steps for all recordings created in Dynamics 365 for operations.
To edit an activity diagram, right-click on a blank area of the flowchart and in the bottom toolbar, select Edit. For more information about BPM flowcharts, see Flowcharts in Business process modeler.
You can associate a business process with a Visio diagram. Typically, this is used for high-level processes that cannot be represented with a Task recording. BPM supports .vsd and .vsdx files, .vsdm files (macros) are not supported. When .vsd files have macros, BPM disables the execution of the macro.
To publish and distribute your BPM library to other organizations, typically customers that are implementing Dynamics 365 for Operations complete the following steps.
This will make the library available to all users that are part of your organization, which would be all users who sign in to LCS using your organization’s domain. For example, all users with a @contoso.com account.
The implementation stage of a project starts by synchronizing a BPM library with your Visual Studio Team Services (VSTS) project. The enables you to review processes and associate requirements with business processes. It also enables you to track the progress of your implementation project in VSTS and associate work items like bugs, tasks, backlog items, tests, and documents with requirements and business processes.
To learn more about VSTS, visit www.visualstudio.com/team-services.
If you have already setup Visual Studio Team Services from LCS, you can skip this section.
To connect to a VSTS project, LCS is authenticated by using a personal access token. Use the following steps to create a personal access token in VSTS.
After you have set up the connection between the LCS project and a VSTS project, go to the Business process libraries page, select the ellipsis (…) of the library that you want to synchronize, and select VSTS sync.
You can also start VSTS sync from the toolbar of a BPM library.
If you want to cancel sync, go back to the same menu.
When you synchronize a BPM library with a VSTS project, a VSTS work item is created for each business process line in the BPM library. In addition, the hierarchy of business processes in BPM is reflected in the hierarchy of work items in VSTS. The type of work items created in VSTS depends on your LCS project settings.
This is a one-way sync, changes in LCS are reflected in VSTS but not the other way around.
The following is synchronized:
During the requirements gathering phase of a project, use the BPM library to review business processes and tasks, and to identify requirements. In BPM, you can mark business processes as Reviewed to track the review process.
To mark a process or sub process, as reviewed, select the process in BPM, and then select Mark as reviewed in the Overview pane.
When a business process is marked as reviewed, the Reviewed column is updated.
The Reviewed column shows the following:
If the BPM library is synchronized with a VSTS project, when you mark a process as reviewed its state changes to Active in VSTS.
While reviewing a business process that is connected to VSTS, you can add a requirement directly to your VSTs project.
This will create a requirement work item in VSTS associated with the current business process.
On the Requirements tab, you can also navigate to the VSTS work items associated with the current business process by selecting the appropriate links.
The latest version of AF 1.1 CU (cumulative update) 8 was released on 12/7/2016, and available on https://www.microsoft.com/en-us/download/details.aspx?id=54440. The detailed hotfix information is listed on https://support.microsoft.com/en-us/kb/3199763.
It has fix for the following three types of issues:
1. Fixed a memory leak that occurs in specific scenarios. When a dependent service of AppFabric is unresponsive, this results in undisposed objects that accumulate over time. This issue is more noticeable in scenarios where the dependent services have regularly scheduled downtimes that aren’t synchronized with the AppFabric server’s downtime.
2. Fixed an intermittent crash issue. The scenario requires that the AppFabric server has several dependent services and many persisted workflows. The crash is rare but typically occurs during startup.
3. Fixed a crash in the AppFabric Event Collection Service when ETW events that are associated with Workflow Versioning are present. If the Workflow instances that are monitored in the AppFabric Dashboard use Workflow Versioning, ETW events that the Event Collection Service doesn’t recognize may be generated. This causes the Event Collection Service to crash.
For workflow versioning scenario (i.e. any new workflow feature .NET 4.5 onwards), AppFabric dashboard was not reporting all the events. It has been addressed with the current CU release.
The above hotfix contains fix from the AppFabric point of view (i.e. itself ready to handle the new events). However, the AF associated database (SQL) needs to be updated with new set of SQL scripts, which comes in the folder C:Program FilesAppFabric 1.1 for Windows ServerSchema (in general). Schema folder is updated for only one file – Create_Monitoring_Schema.sql. This has the trigger definition script updated for ASWfEventsTable_AfterInsertTrigger, that allows tracking additional events.
Note: You would have to run the SQL script Create_Monitoring_Schema.sql explicitly once CU 8 (kb/3199763) is installed (for additional events tracked in AF dashboard).
I hope this helps!
One of my favorite features in VS2015 is the Roslyn based live static code analyzers Interactive IDE feedback on my code gives me ‘opportunities for improvement’ to my code. Think FxCop++ without having to build your project. I like to say “in your face” developer education. See “bad things” and fix them before some other developer makes fun of you! For those not familiar with Roslyn Analyzers I will suggest you view Falafel’s blog post on the topic at https://blog.falafel.com/live-code-analysis-day-20-visual-studio-2015/ . He gives a great overview with examples of the toolset inside VS2015.
Well now VS2017 RC3 is out on the wire. I setup an Azure VM and install VS2017R3 into it. Futzing around, I noticed working with VS2017RC3 that there were a number of new analyzers. I attempted to find a comprehensive list of analyzers added to or removed from the stock set that VS ships with. Unable to find a list of deltas, I manually perused the “all rules” list, and noted the following changes below. I share them with you for reference.
**** Warning – This is a very unscientific and manual analysis and may change from time to time ****
If you find any omissions or extra additions, please let me know!
Microsoft.CodeAnalysis.CSharp
[-] CS0028 Method has the wrong signature to be an entry point
[-] CS0402 An entry point cannot be generic or in a generic type
[-] CS7022 The entry point of the program is global script code
[+] CS8123 The tuple element name is ignored because a different name is specified by the assignment target
Microsoft.CodeAnalysis.CSharp.EditorFeatures
[+] IDE0007 Use implicit type
[+] IDE0008 Use explicit type
[+] IDE0009 Member access should be qualified
[+] IDE0011 Add braces
[+] IDE0012 Simplify Names
[+] IDE0013 Simplify Names
[+] IDE0014 Use framework type
[+] IDE0015 Use framework type
[+] IDE0016 Use ‘throw’ expression
[+] IDE0017 Simplify object initialization
[+] IDE0018 Inline variable declaration
[+] IDE0019 Use pattern matching
[+] IDE0020 Use pattern matching
[+] IDE0021 Use expression body for methods
[+] IDE0022 Use expression body for operators
[+] IDE0023 Use expression body for operators
[+] IDE0024 Use expression body for properties
[+] IDE0025 Use expression body for indexers
[+] IDE0026 Use expression body for accessors
[+] IDE0027 Simplify colection initialization
[+] IDE0028 Use coalesce expression
[+] IDE0029 Use coalesce expression
[+] IDE0030 Use null propogation
[+] IDE1005 Delegate invocation can be simplified
[+] IDE1006 Naming Styles
Microsoft.CodeAnalysis.Features
[+] IDE0010 Add missing cases
[+] IDE0032 Use explicity provided tuple name
Microsoft.Rules.Managed
[-] CA2123 Override link demands should be identical to base
[-] CA2130: Security critical constants should be transparent
[-] CA2131: Security critical types may not participate in type equivalence
[-] CA2132: Default constructors must be at least as critical as base type default constructors
[-] CA2133: Delegates must bind to methods with consistent transparency
[-] CA2134: Methods must keep consistent transparency when overriding base methods
[-] CA2135: Level 2 assemblies should not contain LinkDemands
[-] CA2136: Members should not have conflicting transparency annotations
[-] CA2137: Transparent methods must contain only verifiable IL
[-] CA2138: Transparent methods must not call methods with the SuppressUnmanagedCodeSecurity attribute
[-] CA2139: Transparent methods may not use the HandleProcessCorruptingExceptions attribute
[-] CA2140: Transparent code must not reference security critical items
[-] CA2141:Transparent methods must not satisfy LinkDemands
[-] CA2142: Transparent code should not be protected with LinkDemands
[-] CA2143: Transparent methods should not use security demands
[-] CA2144: Transparent code should not load assemblies from byte arrays
[-] CA2145: Transparent methods should not be decorated with the SuppressUnmanagedCodeSecurityAttribute
[-] CA2146: Types must be at least as critical as their base types and interfaces
[-] CA2147: Transparent methods may not use security asserts
[-] CA2149: Transparent methods must not call into native code
[-] CA2151: Fields with critical types should be security critical.
[-] CA5122: P/Invoke declarations should not be safe-critical
Microsoft.VisualStudio.Azure.Fabric.DiagnosticAnalyzers
[+] SF1001 The name of a service type has not been consistently applied in all places
[+] SF1002 Assembly name and serviceentry point are inconsistent across project and manifest
Enjoy!
Deep learning is impacting everything from healthcare to transportation to manufacturing, and more. Companies are turning to deep learning to solve hard problems, like image classification, speech recognition, object recognition, and machine translation.
There are many popular frameworks, including Microsoft Cognitive Toolkit, Tensorflow, MXNet, Theano, etc. Caffe is one of the most famous non-symbolic (imperative) neural network frameworks, and widely used in many areas including computer vision. Furthermore, CaffeOnSpark combines Caffe with Apache Spark, in which case deep learning can be easily used on an existing Hadoop cluster together with Spark ETL pipelines, reducing system complexity and latency for end-to-end learning.
HDInsight is the only fully-managed cloud Hadoop offering that provides optimized open source analytic clusters for Spark, Hive, MapReduce, HBase, Storm, Kafka, and R Server backed by a 99.9% SLA. Each of these big data technologies and ISV applications are easily deployable as managed clusters with enterprise-level security and monitoring.
Some users are asking us about how to use deep learning on HDInsight, which is Microsoft’s PaaS Hadoop product. We will have more to share in the future, but today we want to summarize a technical blog on how to use Caffe on HDInsight Spark.
If you have installed Caffe before, you will notice that installing this framework is a little bit challenging. In this blog, we will first illustrate how to install Caffe on Spark for an HDInsight cluster, then use the built-in MNIST demo to demostrate how to use Distributed Deep Learning using HDInsgiht Spark on CPUs.
There are four major steps to get it work on HDInsight.
Since HDInsight is a PaaS solution, it offers great platform features – so it is quite easy to perform some tasks. One of the features that we heavily use in this blog post is called Script Action, with which you can execute shell commands to customize cluster nodes (head node, worker node, or edge node).
To get started, we need to install the dependencies we need. The Caffe site and CaffeOnSpark site offers some very useful wiki for installing the dependencies for Spark on YARN mode (which is the mode for HDInsight Spark), but we need to add a few more dependencies for HDInsight platform. We will use the script action as below and run it on all the head nodes and worker nodes. This script action will take about 20 minutes, as those dependencies also depend on other packages. I put the script in my GitHub location so it is accessible by the cluster.
#!/bin/bash
#Please be aware that installing the below will add additional 20 mins to cluster creation because of the dependencies
#installing all dependencies, including the ones mentioned in http://caffe.berkeleyvision.org/install_apt.html, as well a few packages that are not included in HDInsight, such as gflags, glog, lmdb, numpy
#It seems numpy will only needed during compilation time, but for safety purpose we install them on all the nodes
sudo apt-get install -y libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler maven libatlas-base-dev libgflags-dev libgoogle-glog-dev liblmdb-dev build-essential libboost-all-dev python-numpy python-scipy python-matplotlib ipython ipython-notebook python-pandas python-sympy python-nose
#install protobuf
wget https://github.com/google/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.gz
sudo tar xzvf protobuf-2.5.0.tar.gz -C /tmp/
cd /tmp/protobuf-2.5.0/
sudo ./configure
sudo make
sudo make check
sudo make install
sudo ldconfig
echo "protobuf installation done"
There are two steps in the script action above. The first step is to install all the required libraries. Those libraries include the necessary libraries for both compiling Caffe(such as gflags, glog) and running Caffe (such as numpy). We are using libatlas for CPU optimization, but you can always follow the CaffeOnSpark wiki on installing other optimization libraries, such as MKL or CUDA (for GPU).
The second step is to download, compile, and install protobuf 2.5.0 for Caffe during runtime. Protobuf 2.5.0 is required, however this version is not available as a package on Ubuntu 16, so we need to compile it from the source code. There are also a few resources on the Internet on how to compile it, such as this
To simply get started, you can just run this script action against your cluster to all the worker nodes and head nodes (for HDInsight 3.5). You can either run the script actions for a running cluster, or you can also run the script actions during the cluster provision time. For more details on the script actions, please see the documentation here
The second step is to build Caffe on the headnode, and then distribute the compiled libraries to all the worker nodes. In this step, you will need to ssh into your headnode, then simply follow the CaffeOnSpark build process, and below is the script I use to build CaffeOnSpark with a few additional steps.
#!/bin/bash
git clone https://github.com/yahoo/CaffeOnSpark.git --recursive
export CAFFE_ON_SPARK=$(pwd)/CaffeOnSpark
pushd ${CAFFE_ON_SPARK}/caffe-public/
cp Makefile.config.example Makefile.config
echo "INCLUDE_DIRS += ${JAVA_HOME}/include" >> Makefile.config
#Below configurations might need to be updated based on actual cases. For example, if you are using GPU, or using a different BLAS library, you may want to update those settings accordingly.
echo "CPU_ONLY := 1" >> Makefile.config
echo "BLAS := atlas" >> Makefile.config
echo "INCLUDE_DIRS += /usr/include/hdf5/serial/" >> Makefile.config
echo "LIBRARY_DIRS += /usr/lib/x86_64-linux-gnu/hdf5/serial/" >> Makefile.config
popd
#compile CaffeOnSpark
pushd ${CAFFE_ON_SPARK}
#always clean up the environment before building (especially when rebuiding), or there will be errors such as "failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (proto) on project caffe-distri: An Ant BuildException has occured: exec returned: 2"
make clean
#the build step usually takes 20~30 mins, since it has a lot maven dependencies
make build
popd
export LD_LIBRARY_PATH=${CAFFE_ON_SPARK}/caffe-public/distribute/lib:${CAFFE_ON_SPARK}/caffe-distri/distribute/lib
hadoop fs -mkdir -p wasb:///projects/machine_learning/image_dataset
${CAFFE_ON_SPARK}/scripts/setup-mnist.sh
hadoop fs -put -f ${CAFFE_ON_SPARK}/data/mnist_*_lmdb wasb:///projects/machine_learning/image_dataset/
${CAFFE_ON_SPARK}/scripts/setup-cifar10.sh
hadoop fs -put -f ${CAFFE_ON_SPARK}/data/cifar10_*_lmdb wasb:///projects/machine_learning/image_dataset/
#put the already compiled CaffeOnSpark libraries to wasb storage, then read back to each node using script actions. This is because CaffeOnSpark requires all the nodes have the libarries
hadoop fs -mkdir -p /CaffeOnSpark/caffe-public/distribute/lib/
hadoop fs -mkdir -p /CaffeOnSpark/caffe-distri/distribute/lib/
hadoop fs -put CaffeOnSpark/caffe-distri/distribute/lib/* /CaffeOnSpark/caffe-distri/distribute/lib/
hadoop fs -put CaffeOnSpark/caffe-public/distribute/lib/* /CaffeOnSpark/caffe-public/distribute/lib/
I actually do more than what the documentation of CaffeOnSpark says. The changes are:
When I was first trying to build CaffeOnSpark, sometimes it will say
failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (proto) on project caffe-distri: An Ant BuildException has occured: exec returned: 2
Simply clean the code repository by “make clean” and then run “make build” will solve this issue, as long as you have the correct dependencies.
Sometimes maven gives me the connection time out error, similar to below:
Retry:
[INFO] Downloading: https://repo.maven.apache.org/maven2/com/twitter/chill_2.11/0.8.0/chill_2.11-0.8.0.jar
Feb 01, 2017 5:14:49 AM org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://repo.maven.apache.org:443: Connection timed out (Read failed)
It will be OK after I wait for a few minutes and then just rebuild the code, so I suspect it might be Maven somehow limits the traffic from a given IP address.
You probably will see a test failure when doing the final check for CaffeOnSpark, similar with below. This is prabably related with UTF-8 encoding, but should not impact the usage of Caffe
Run completed in 32 seconds, 78 milliseconds.
Total number of tests run: 7
Suites: completed 5, aborted 0
Tests: succeeded 6, failed 1, canceled 0, ignored 0, pending 0
*** 1 TEST FAILED ***
The next step is to distribute the libraries (basically the libraries in CaffeOnSpark/caffe-public/distribute/lib/ and CaffeOnSpark/caffe-distri/distribute/lib/) to all the nodes. In Step 2, we put those libraries on BLOB storage, and in this step, we will use script actions to copy it to all the head nodes and worker nodes.
To do this, simple run a script action as below (you need to point to the right location specific to your cluster):
#!/bin/bash
hadoop fs -get wasb:///CaffeOnSpark /home/xiaoyzhu/
Because in step 2, we put it on the BLOB storage which is accessible to all the nodes, in this step we just simply copy it to all the nodes.
After running the above steps, Caffe is alreay installed on the headnode and we are good to go. The next step is to write a Caffe model.
Caffe is using an “expressive architecture”, where for composing a model, you just need to define a configuration file, and without coding at all (in most cases). So let’s take a look there.
The model we will train today is a sample model for MNIST training. The MNIST database of handwritten digits has a training set of 60,000 examples, and a test set of 10,000 examples. It is a subset of a larger set available from NIST. The digits have been size-normalized and centered in a fixed-size image. CaffeOnSpark has some scripts to download the dataset and convert it into the right format.
CaffeOnSpark provides some network topologies example for MNIST training. It has a nice design of splitting the network architecture (the topology of the network) and optimization. In this case, There are two files required:
the “Solver” file (${CAFFE_ON_SPARK}/data/lenet_memory_solver.prototxt) is used for overseeing the optimization and generating parameter updates. For example, it defines whether CPU or GPU will be used, what’s the momentum, how many iterations will be, etc. It also defies which neuron network topology should the program use (which is the second file we need). For more information about Solver, please refer to Caffe documentation.
For this example, since we are using CPU rather than GPU, we should change the last line to:
# solver mode: CPU or GPU
solver_mode: CPU
You can change other lines as needed.
The second file (${CAFFE_ON_SPARK}/data/lenet_memory_train_test.prototxt) defines how the neuron network looks like, and the relevant input and output file. We also need to update the file to reflect the training data location. Change the following part in lenet_memory_train_test.prototxt (you need to point to the right location specific to your cluster):
For more information on how to define the network, please check the Caffe documentation on MNIST dataset
For the purpose of this blog, we just use this simple MNIST example. You should run the command below from the head node:
spark-submit --master yarn --deploy-mode cluster --num-executors 8 --files ${CAFFE_ON_SPARK}/data/lenet_memory_solver.prototxt,${CAFFE_ON_SPARK}/data/lenet_memory_train_test.prototxt --conf spark.driver.extraLibraryPath="${LD_LIBRARY_PATH}" --conf spark.executorEnv.LD_LIBRARY_PATH="${LD_LIBRARY_PATH}" --class com.yahoo.ml.caffe.CaffeOnSpark ${CAFFE_ON_SPARK}/caffe-grid/target/caffe-grid-0.1-SNAPSHOT-jar-with-dependencies.jar -train -features accuracy,loss -label label -conf lenet_memory_solver.prototxt -devices 1 -connection ethernet -model wasb:///mnist.model -output wasb:///mnist_features_result
Basically it distributes the required files (lenet_memory_solver.prototxt and lenet_memory_train_test.prototxt) to each YARN container, and also set the relevant PATH of each Spark driver/executor to LD_LIBRARY_PATH, which is defined in the previous code snippet and points to the location that has CaffeOnSpark libraries.
Since we are using YARN cluster mode, in which case the Spark driver will be scheduled to an arbitrary container (and an arbitrary worker node) you should only see in the console outputting something like:
17/02/01 23:22:16 INFO Client: Application report for application_1485916338528_0015 (state: RUNNING)
If you want to know what happened, you usually need to get the Spark driver’s log, which has more information. In this case, you need to go to the YARN UI to find the relevant YARN logs. You can get the YARN UI by this URL:
https://yourclustername.azurehdinsight.net/yarnui
You can take a look at how many resources are allocated for this particular application. You can click the “Scheduler” link, and then you will see that for this application, there are 9 containers running. We ask YARN to provide 8 executors, and another container is for driver process.
You may want to check the driver logs or container logs if there are failures. For driver logs, you can click the application ID in YARN UI, then click the “Logs” button. The driver logs are written into stderr.
For example, you might see some of the error below from the driver logs, indicating you allocate too many executors.
17/02/01 07:26:06 ERROR ApplicationMaster: User class threw exception: java.lang.IllegalStateException: Insufficient training data. Please adjust hyperparameters or increase dataset.
java.lang.IllegalStateException: Insufficient training data. Please adjust hyperparameters or increase dataset.
at com.yahoo.ml.caffe.CaffeOnSpark.trainWithValidation(CaffeOnSpark.scala:261)
at com.yahoo.ml.caffe.CaffeOnSpark$.main(CaffeOnSpark.scala:42)
at com.yahoo.ml.caffe.CaffeOnSpark.main(CaffeOnSpark.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:627)
Sometimes, the issue can happen in executors rather than drivers. In this case, you need to check the container logs. You can always get the driver container logs, and then get the failed container. For example, I met this failure when I was running Caffe.
17/02/01 07:12:05 WARN YarnAllocator: Container marked as failed: container_1485916338528_0008_05_000005 on host: 10.0.0.14. Exit status: 134. Diagnostics: Exception from container-launch.
Container id: container_1485916338528_0008_05_000005
Exit code: 134
Exception message: /bin/bash: line 1: 12230 Aborted (core dumped) LD_LIBRARY_PATH=/usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/home/xiaoyzhu/CaffeOnSpark/caffe-public/distribute/lib:/home/xiaoyzhu/CaffeOnSpark/caffe-distri/distribute/lib /usr/lib/jvm/java-8-openjdk-amd64/bin/java -server -Xmx4608m '-Dhdp.version=' '-Detwlogger.component=sparkexecutor' '-DlogFilter.filename=SparkLogFilters.xml' '-DpatternGroup.filename=SparkPatternGroups.xml' '-Dlog4jspark.root.logger=INFO,console,RFA,ETW,Anonymizer' '-Dlog4jspark.log.dir=/var/log/sparkapp/${user.name}' '-Dlog4jspark.log.file=sparkexecutor.log' '-Dlog4j.configuration=file:/usr/hdp/current/spark2-client/conf/log4j.properties' '-Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl' -Djava.io.tmpdir=/mnt/resource/hadoop/yarn/local/usercache/xiaoyzhu/appcache/application_1485916338528_0008/container_1485916338528_0008_05_000005/tmp '-Dspark.driver.port=43942' '-Dspark.history.ui.port=18080' '-Dspark.ui.port=0' -Dspark.yarn.app.container.log.dir=/mnt/resource/hadoop/yarn/log/application_1485916338528_0008/container_1485916338528_0008_05_000005 -XX:OnOutOfMemoryError='kill %p' org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url spark://CoarseGrainedScheduler@10.0.0.13:43942 --executor-id 4 --hostname 10.0.0.14 --cores 3 --app-id application_1485916338528_0008 --user-class-path file:/mnt/resource/hadoop/yarn/local/usercache/xiaoyzhu/appcache/application_1485916338528_0008/container_1485916338528_0008_05_000005/__app__.jar > /mnt/resource/hadoop/yarn/log/application_1485916338528_0008/container_1485916338528_0008_05_000005/stdout 2> /mnt/resource/hadoop/yarn/log/application_1485916338528_0008/container_1485916338528_0008_05_000005/stderr
Stack trace: ExitCodeException exitCode=134: /bin/bash: line 1: 12230 Aborted (core dumped) LD_LIBRARY_PATH=/usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/home/xiaoyzhu/CaffeOnSpark/caffe-public/distribute/lib:/home/xiaoyzhu/CaffeOnSpark/caffe-distri/distribute/lib /usr/lib/jvm/java-8-openjdk-amd64/bin/java -server -Xmx4608m '-Dhdp.version=' '-Detwlogger.component=sparkexecutor' '-DlogFilter.filename=SparkLogFilters.xml' '-DpatternGroup.filename=SparkPatternGroups.xml' '-Dlog4jspark.root.logger=INFO,console,RFA,ETW,Anonymizer' '-Dlog4jspark.log.dir=/var/log/sparkapp/${user.name}' '-Dlog4jspark.log.file=sparkexecutor.log' '-Dlog4j.configuration=file:/usr/hdp/current/spark2-client/conf/log4j.properties' '-Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl' -Djava.io.tmpdir=/mnt/resource/hadoop/yarn/local/usercache/xiaoyzhu/appcache/application_1485916338528_0008/container_1485916338528_0008_05_000005/tmp '-Dspark.driver.port=43942' '-Dspark.history.ui.port=18080' '-Dspark.ui.port=0' -Dspark.yarn.app.container.log.dir=/mnt/resource/hadoop/yarn/log/application_1485916338528_0008/container_1485916338528_0008_05_000005 -XX:OnOutOfMemoryError='kill %p' org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url spark://CoarseGrainedScheduler@10.0.0.13:43942 --executor-id 4 --hostname 10.0.0.14 --cores 3 --app-id application_1485916338528_0008 --user-class-path file:/mnt/resource/hadoop/yarn/local/usercache/xiaoyzhu/appcache/application_1485916338528_0008/container_1485916338528_0008_05_000005/__app__.jar > /mnt/resource/hadoop/yarn/log/application_1485916338528_0008/container_1485916338528_0008_05_000005/stdout 2> /mnt/resource/hadoop/yarn/log/application_1485916338528_0008/container_1485916338528_0008_05_000005/stderr
at org.apache.hadoop.util.Shell.runCommand(Shell.java:933)
at org.apache.hadoop.util.Shell.run(Shell.java:844)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1123)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:225)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:317)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:83)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 134
In this case, you need to get the failed container ID (in the above case, it is container_1485916338528_0008_05_000005). Then you need to run
yarn logs -containerId container_1485916338528_0008_03_000005
from the headnode. After checking container failure, I realize that I use GPU mode because I forget to change the mode in lenet_memory_solver.prototxt.
17/02/01 07:10:48 INFO LMDB: Batch size:100
WARNING: Logging before InitGoogleLogging() is written to STDERR
F0201 07:10:48.309725 11624 common.cpp:79] Cannot use GPU in CPU-only Caffe: check mode.
Since we are allocating 8 executors, and the network topology is simple, it should only take around 30 minutes to run the result. From the command line, you can see that we put the model to wasb:///mnist.model, and put the results to a folder named wasb:///mnist_features_result.
You can get the results by running
hadoop fs -cat hdfs:///mnist_features_result/*
and the result looks like:
{"SampleID":"00009597","accuracy":[1.0],"loss":[0.028171852],"label":[2.0]}
{"SampleID":"00009598","accuracy":[1.0],"loss":[0.028171852],"label":[6.0]}
{"SampleID":"00009599","accuracy":[1.0],"loss":[0.028171852],"label":[1.0]}
{"SampleID":"00009600","accuracy":[0.97],"loss":[0.0677709],"label":[5.0]}
{"SampleID":"00009601","accuracy":[0.97],"loss":[0.0677709],"label":[0.0]}
{"SampleID":"00009602","accuracy":[0.97],"loss":[0.0677709],"label":[1.0]}
{"SampleID":"00009603","accuracy":[0.97],"loss":[0.0677709],"label":[2.0]}
{"SampleID":"00009604","accuracy":[0.97],"loss":[0.0677709],"label":[3.0]}
{"SampleID":"00009605","accuracy":[0.97],"loss":[0.0677709],"label":[4.0]}
The SampleID represents the ID in the MNIST dataset, and the label is the number that the model identifies.
In this blog, I walk you through installing CaffeOnSpark and we run a simple example. HDInsight is a full managed cloud distributed compute platform, and is the best place for running machine learning and advanced analytics workloads on large data set, such as R Server on HDInsight, Spark ML, etc. For Distributed Deep Learning, we just demonstrated the ability to run Caffe on HDInsgiht Spark, and we will have more to share in the future.
Feel free to drop any comments and feedback to xiaoyzhu at microsoft dot com, and I have some sample scripts and the most up-to-date version on GitHub.
In November 2016, we announced availability of Visio Online Preview, a new way to view and share your Visio diagrams on the web. Today, we’re making it even easier for our customers to use Visio Online Preview, by adding support for VSD file format and new cloud storage options for storing and viewing Visio diagrams on the web.
When Visio Online Preview was launched, users were able to view their .vsdx files on the web, which is the latest and default file format for Visio 2013 and onwards. Now, you will also be able to view .vsd files right in your browser. VSD file format is the default file open and save format for Visio 2003, 2007 and 2010.
This update is applicable to all Office 365 customers.
Earlier, Visio Online Preview was available for users whose diagrams are stored in SharePoint Online and OneDrive for Business.
We are extending this capability to users who are using Exchange Online, Outlook.com or Hotmail for their email. With Visio Online Preview, you can now view attached Visio files directly in the browser. Also, from now on, any partner in the Cloud Storage Partner Program (CSPP) can enable viewing of Visio files for their application.
We’ll continue to add more capabilities to Visio Online over time. Do share your feedback and requests at UserVoice.
Rahul Mittal
Program Manager 2
Microsoft Visio Team