tag:blogger.com,1999:blog-51795849502191171072024-03-05T11:51:41.939+00:00Adrian's BlogMy personal programming blog, with a hint of petroleum.Adrian L Thomashttp://www.blogger.com/profile/05314909861645693237noreply@blogger.comBlogger14125tag:blogger.com,1999:blog-5179584950219117107.post-39458246577871536042016-12-01T22:59:00.001+00:002016-12-01T23:00:39.466+00:00Error when installing Docker on Windows Server 2016When following the install guide <a href="https://msdn.microsoft.com/en-us/virtualization/windowscontainers/quick_start/quick_start_windows_server" target="_blank">here</a>, ensure that you are running PowerShell as x64 (not x86!) otherwise you'll get the following error when trying to run the command:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">PS C:\Windows\system32> Install-Package -Name docker -ProviderName DockerMsftProvider</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">The package(s) come(s) from a package source that is not marked as trusted.</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">Are you sure you want to install software from 'DockerDefault'?</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "N"): Y</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">Install-Package : The term 'Get-WindowsFeature' is not recognized as the name of a cmdlet, function, script file, or</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">again.</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">At line:1 char:1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">+ Install-Package -Name docker -ProviderName DockerMsftProvider</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> + CategoryInfo : InvalidOperation: (Microsoft.Power....InstallPackage:InstallPackage) [Install-Package],</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> Exception</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> + FullyQualifiedErrorId : FailedToDownload,Install-Package,Microsoft.PowerShell.PackageManagement.Cmdlets.InstallP</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> ackage</span>Adrian L Thomashttp://www.blogger.com/profile/05314909861645693237noreply@blogger.com0tag:blogger.com,1999:blog-5179584950219117107.post-58987914241545527432016-06-17T17:14:00.001+01:002016-06-17T17:17:32.342+01:00Owin Startup Not ExecutedI've been bashing my head over this problem today. My Owin Startup class wasn't being executed. I'd followed the usual instructions of ensuring the "<span style="font-family: Courier New, Courier, monospace;">Microsoft.Owin.Host.SystemWeb</span>" package was installed. I had the correct attribute on the assembly (<span style="font-family: Courier New, Courier, monospace;">[assembly: OwinStartup(typeof(StartupDemo.TestStartup))]</span>), clearing the ASP.NET cache, etc etc.<br />
<br />
Eventually I found that if you have set a friendly name for your startup class (in the instance where you may have multiple Startup.cs files) - then you need to also make sure you add the friendly name to your appSettings in the web.config too!<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"><add key="owin:appStartup" value="StartupDemo.ProductionStartup"/></span><br />
<br />
Doh!Adrian L Thomashttp://www.blogger.com/profile/05314909861645693237noreply@blogger.com0tag:blogger.com,1999:blog-5179584950219117107.post-47796749167226683552015-12-14T15:52:00.002+00:002015-12-14T16:31:05.153+00:00Feature Folders in ASP.NET 5 (RC1) with MVC 6 Feature folders are a great way of structuring an MVC project in order to quickly and easily find the files for areas of your website.<br />
<br />
You can turn a typical structure like this:<br />
<br />
<pre><code>- Website
+ Content
+ Controllers
+ Models
+ Views
</code></pre>
<br />
in to this:<br />
<pre><code>- Website
- Features
- Portfolio
Index.cshtml
Index.js
IndexViewModel.cs
</code></pre>
<br />
This has been possible for a while now, however the API has changed a bit with ASP.NET 5 (as well as all the beta versions). Currently in ASP.NET 5 RC1 this is how you do it:<br />
<br />
Create a new view engine, something like:<br />
<br />
<pre><code>FeatureViewLocationRazorViewEngine.cs
public class FeatureViewLocationRazorViewEngine : RazorViewEngine
{
private readonly string[] _featureFolderViewLocationFormats;
public FeatureViewLocationRazorViewEngine(IRazorPageFactory pageFactory, IRazorViewFactory viewFactory, IOptions<RazorViewEngineOptions> options, IViewLocationCache viewLocationCache)
:base(pageFactory, viewFactory, options, viewLocationCache)
{
_featureFolderViewLocationFormats = new[]
{
"~/Features/{1}/{0}.cshtml",
"~/Features/{1}/{0}.vbhtml",
"~/Features/Shared/{0}.cshtml",
"~/Features/Shared/{0}.vbhtml"
};
}
public override IEnumerable<string> ViewLocationFormats => _featureFolderViewLocationFormats;
}
</code></pre>
Then modify the startup file:
<br />
<pre><code>Startup.cs
public void ConfigureServices(IServiceCollection services)
{
services.AddSingleton<IRazorViewEngine, FeatureViewLocationRazorViewEngine>();
services.AddMvc();
}
</code></pre>
That's it!Adrian L Thomashttp://www.blogger.com/profile/05314909861645693237noreply@blogger.com0tag:blogger.com,1999:blog-5179584950219117107.post-14160064489917472302014-12-11T22:17:00.000+00:002014-12-11T22:18:09.322+00:00Navigating away from a page in Windows Phone 8.1 with Prism throws a System.ArgumentNullExceptionI just had a little headache with something that should be fairly trivial, but unfortunately the exception & stack trace weren't particularly descriptive. I was navigating from one page to another in my universal app. The navigation worked for Windows, but not for Phone, and I got the following:<br />
<br />
<pre>{System.ArgumentNullException: Value cannot be null.
Parameter name: key
at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
at System.Collections.Generic.Dictionary`2.set_Item(TKey key, TValue value)
at Microsoft.Practices.Prism.StoreApps.VisualStateAwarePage.OnNavigatedFrom(NavigationEventArgs e)}
</pre>
<pre>
</pre>
<pre></pre>
<pre><span style="font-family: Times New Roman;"><span style="white-space: normal;">Well it turns out when you create a new Windows Phone XAML page in Visual Studio, it kindly overrides the OnNavigatedTo method in the code behind (of course I didn't look here, I'm using MVVM!), which seems to throw Prism out of whack. Remove the overriden method (or just call it's base) and navigation succeeds, yippee.</span></span></pre>
Adrian L Thomashttp://www.blogger.com/profile/05314909861645693237noreply@blogger.com2tag:blogger.com,1999:blog-5179584950219117107.post-35791776235262374942014-10-05T00:41:00.000+01:002014-10-05T00:41:08.531+01:00Visual Studio 2014 CTP Crashes on Get Latest when using a touch screenSo, just a quick one. I downloaded Visual Studio 2014 CTP and tried to get some code from source control. Unfortunately, Visual Studio crashes without warning. A quick look in the event viewer shows that it's a touch related issue (null reference exception). I installed VS on my Surface Pro, and when using the touch pad it seems to work fine.<br />
<br />
<br />
<pre>
Application: devenv.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.NullReferenceException
Stack:
at System.Windows.Input.StylusLogic.IsTouchStylusDevice(System.Windows.Input.StylusDevice)
</pre>
(Complete stacktrace omitted)
Looks like I'll be sticking with the trackpad for now :-)Adrian L Thomashttp://www.blogger.com/profile/05314909861645693237noreply@blogger.com0tag:blogger.com,1999:blog-5179584950219117107.post-55049378340816783092014-09-28T10:52:00.002+01:002017-03-06T20:21:13.221+00:00Introduction to ASP.NET MVCI did a talk for Black Marble's Friday training last month. I recently passed a Microsoft certification on MVC and it seemed appropriate to talk about what I had learnt. There are also some tips on Microsoft exams.<br />
<br />
<div style="text-align: center;">
<iframe allowfullscreen="" frameborder="0" height="270" src="//www.youtube.com/embed/um2rUS6EqZA" width="480"></iframe><br />
<br />
<div style="text-align: left;">
The slides are <a href="https://github.com/AdrianLThomas/Blog/raw/master/Assets/Blogger/2014/FridayTraining/ASP.NET%20MVC%20-%20Adrian%20L%20Thomas.pptx" target="_blank">available here</a></div>
</div>
Adrian L Thomashttp://www.blogger.com/profile/05314909861645693237noreply@blogger.com0tag:blogger.com,1999:blog-5179584950219117107.post-55599933309025585132014-03-14T19:42:00.003+00:002017-03-06T20:17:14.423+00:00Diagnosing a lean condition using VCDS-LiteI recently went on a rolling road day to get an idea of my current cars performance (Seat Leon Cupra R, 1.8T), and was happy to find it wasn't too down on power for a 9 year old car.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://raw.githubusercontent.com/AdrianLThomas/Blog/master/Assets/Blogger/2014/automotive/14_03_2014/DynoRun.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="464" src="https://raw.githubusercontent.com/AdrianLThomas/Blog/master/Assets/Blogger/2014/automotive/14_03_2014/DynoRun.jpg" width="640" /></a></div>
<br />
<br />
Unfortunately I was (ill-) advised that the engine was running lean (the air to fuel ratio was too high). There are several areas to look at when diagnosing this, but the common one tends to be an air leak in the crankcase ventilation system.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://raw.githubusercontent.com/AdrianLThomas/Blog/master/Assets/Blogger/2014/automotive/14_03_2014/PCVSystem.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="396" src="https://raw.githubusercontent.com/AdrianLThomas/Blog/master/Assets/Blogger/2014/automotive/14_03_2014/PCVSystem.jpg" width="640" /></a></div>
<br />
<br />
I sprayed all the pipes in the system with (flammable) electrical contact cleaner. If there is a leak, then the engine will ingest some of the fluid into the engine via the leak and it will combust, creating an audible RPM change. Unfortunately this didn't help in diagnosing, but did lead me on to purchasing a VCDS-Lite Registration ($99 / £60) that enabled me to use my KKL VAG-COM cable I got from Amazon for around £6. The free version does some basic stuff such as reading error codes, but the registered version provides extended logging capabilities.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://raw.githubusercontent.com/AdrianLThomas/Blog/master/Assets/Blogger/2014/automotive/14_03_2014/KKL-VAGCOM.jpeg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://raw.githubusercontent.com/AdrianLThomas/Blog/master/Assets/Blogger/2014/automotive/14_03_2014/KKL-VAGCOM.jpeg" /></a></div>
<br />
<br />
By measuring blocks 001 (RPM) and 031 (Lambda Control Actual & Specification) it provided enough data to calculate the air/fuel ratio requested by the ECU and the actual fuelling measured by the lambda control sensor. Just to note, a faulty lambda sensor is usually easy to spot, as the figures provided by it are either flat-lining or erratic. The figure must be multiplied by 14.7 (stoichiometric) to calculate the AFR.<br />
<br />
Anyway, after consulting with one of the tuners on SeatCupra.net, the 1.8T BAM engines typically run more lean on the standard ECU map than other cars do, which is the only reason I can think of why I was ill-advised to the running of the engine. To confirm this, I used VCDS to to log the above measuring blocks (in 3rd gear, from 2500rpm to 6800rpm) to log the data required. As you can see in the graph below, the fuelling is almost exactly as the ECU requests (as the manufacturer intended) - and where it differs is where the engine is running richer (lower air fuel ratio, therefore more fuel is being combusted). Running rich is much less deteritmental to the engine, and whilst it is an increase in fuel consumption, it also means that the fuel keeps the cylinder walls cooler. Running lean can mean detonation, which is very bad.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://raw.githubusercontent.com/AdrianLThomas/Blog/master/Assets/Blogger/2014/automotive/14_03_2014/AFR.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="396" src="https://raw.githubusercontent.com/AdrianLThomas/Blog/master/Assets/Blogger/2014/automotive/14_03_2014/AFR.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
Adrian L Thomashttp://www.blogger.com/profile/05314909861645693237noreply@blogger.com0tag:blogger.com,1999:blog-5179584950219117107.post-79589940870495036452014-03-01T19:46:00.001+00:002022-04-28T13:49:13.290+01:0012 steps to becoming a mouse widowSo a few months ago I was tasked with going by my day without using a mouse. Easy I thought! and it actually wasn't too bad. I knew, as long as I could open the browser and navigate to Google to find the keyboard shortcuts, I would be OK... so here are some of my favourites.<br />
<br />
<h2>
Windows</h2>
<h3>
<b>Reverse Tabbing</b></h3>
So most people probably know they can tab through GUI's, and what not, but did you know you can go back too? Try it... <b>Shift & Tab</b>.<br />
<br />
<h3>
<b>Open an Application as an Adminstrator</b></h3>
Search for an app in the Start Menu and hit <b>Ctrl & Shift & Enter</b>.<br />
<br />
<h3>
<b>Open an Additional Application To What Is Already Open (Windows 8 Special)</b></h3>
So you already have an instance of Remote Desktop open. Want another without loading up the current instance? Then search for the app and hit <b>Shift & Enter</b>.<br />
<br />
<h3>
<b>Right Clicking</b></h3>
<div>
Ever wondered what the key on the bottom right of the keyboard is? The one between the Windows and Right Ctrl key? That's the menu key, and it behaves the same as if you right clicked the mouse on the currently selected item.</div>
<div>
<br /></div>
<h3>
<b>Task Manager</b></h3>
<b>Ctrl & Shift & Escape</b>... no need for Ctrl & Alt & Delete.<br />
<br />
<h3>
<b>Magnifier</b></h3>
<b>Win Key & [+ Key (Addition)]</b><br />
<b><br /></b>
<h3>
<b>Moving Windows</b></h3>
Align to the side of the window: <b>Win Key & Left Arrow / Right Arrow</b><br />
Maximise/Minimise: <b>Win Key & Up Arrow / Down Arrow</b><br />
<b><br /></b>
<h3>
<b>Settings Charms in Windows 8</b></h3>
<div>
Need the settings for the app? The power menu to shut down the PC? Adjust your volume? Find it here with <b>Win Key & I </b></div>
<br />
<h2>
<b>Browser</b></h2>
<div>
Chrome is my main browser, but as far as I'm aware, most other modern browsers support these shortcuts.</div>
<div>
<b>Switching Between Tabs</b></div>
Ctrl & Number of the tab<br />
<br />
<h3>
<b>Closing Tabs</b></h3>
Ctrl & F4<br />
<br />
<h3>
<b>Reopen Last Tab</b></h3>
Ctrl & Shift & T<br />
<br />
<h3>
<b>Jump to address bar</b></h3>
Alt & D<br />
<br />
<br />
While it was good fun, I actually use these shortcuts very frequently and they without a doubt increase my productivity.<br />
<br />
In summary, I think the only thing I had trouble doing was modifying an image in Microsoft Paint (Win Key & R -> pbrush, by the way).Adrian L Thomashttp://www.blogger.com/profile/05314909861645693237noreply@blogger.comtag:blogger.com,1999:blog-5179584950219117107.post-66096336790136612382013-12-17T16:53:00.002+00:002013-12-17T16:56:57.250+00:00Increasing the maximum received message size for WCF with SharePoint 2013Today I was tasked with an issue of fixing a file upload problem. The server was returning a 413 "Request Entity Too Large" message. The file I was trying to upload was around 3MB, but the server refused anything this big. A few problems arose that are worth noting in fixing this issue.<br />
<br />
A bit of Googling brought up the following result: <a href="http://msdn.microsoft.com/en-us/library/ff599489.aspx">http://msdn.microsoft.com/en-us/library/ff599489.aspx</a><br />
<br />
Which in theory works. However it's not as straightforward as the article implies, as a few things were missed:<br />
<ul>
<li>Run the code as farm administrator</li>
<li>Ensure the service name is lower case (this one caught me out..)</li>
<li>Clearing up</li>
</ul>
<h4>
Run the code as farm administrator</h4>
Simply using the code in the MSDN article won't work if you try in your WCF service, you'll get an Access Denied error on SPWebService.Update(). The way I got round this was creating an event receiver on my project Feature that was scoped at the farm level, this allowed me to override FeatureActivated and FeatureDeactivating and run the code with the right permissions.<br />
<br />
<h4>
Ensure the key for the service name is lower case </h4>
There is no mention of this in the MSDN article, but it's very important as otherwise the changes won't take effect. I noticed that all the other keys in SPWebService.ContentService.WcfServiceSettings were lowercase, and it turns out mine has to be too. This should be the same name of your *.svc file under ISAPI.<br />
<br />
<h4>
Clearing up</h4>
Nothing is mentioned about clearing up either, but it needs doing and this caught me out too. Make sure you override the FeatureDeactivating and remove your settings from SPWebService.ContentService.WcfServiceSettings otherwise you will get strange behaviour (like the settings persisting). Also make sure you set the SPWebService.ContentService.MaxReceivedMessageSize back to 0 (the default setting). <br />
<br />
<h4>
The Code</h4>
<pre style="background: rgb(240, 240, 240); border: 1px dashed rgb(204, 204, 204); color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="-ms-word-wrap: normal; color: black;"> public class MyApiEventReceiver : SPFeatureReceiver
{
public override void FeatureActivated(SPFeatureReceiverProperties properties)
{
SPWebService contentService = SPWebService.ContentService;
contentService.ClientRequestServiceSettings.MaxReceivedMessageSize = -1;
SPWcfServiceSettings wcfSettings = new SPWcfServiceSettings();
wcfSettings.MaxReceivedMessageSize = (1024 * 1024) * 15;
contentService.WcfServiceSettings["wizard.svc"] = wcfSettings;
contentService.Update();
}
public override void FeatureDeactivating(SPFeatureReceiverProperties properties)
{
SPWebService contentService = SPWebService.ContentService;
contentService.WcfServiceSettings.Remove("wizard.svc");
contentService.ClientRequestServiceSettings.MaxReceivedMessageSize = 0;
contentService.Update();
}
}
</code></pre>
Adrian L Thomashttp://www.blogger.com/profile/05314909861645693237noreply@blogger.com1tag:blogger.com,1999:blog-5179584950219117107.post-90817615232095210732013-07-23T16:00:00.001+01:002013-07-23T16:00:23.158+01:00Specifying an Assembly's Location with MEFToday I came across a problem with dynamically loading assemblies using MEF, when the assembly being loaded has a dependency on yet another assembly. In my case this was SignalR.<br />
<br />
<div style="text-align: center;">
Assembly Host (.exe) -> Assembly (.dll) -> SignalR (.dll)</div>
<div style="text-align: center;">
<br /></div>
I was typically seeing the error:<br />
<i>Could not load file or assembly 'AssemblyNameHere, PublicKeyToken=xxxxxxxxxxxxxxxxx' or one of its dependencies. The system cannot find the file specified.</i><br />
<i><br /></i>
The reason for this is because the CLR is looking for the dependency in the directory in which the assembly host is running, and not the location of the assembly MEF loaded in.<br />
<br />
The way around this is in this <a href="http://msdn.microsoft.com/en-us/library/4191fzwb.aspx" target="_blank">MSDN article</a>. By using the probing tag in the app.config, you can specify the directory to look for dependencies. The caveat with using probing is you can only specify sub-directories of the assembly host, so if you want to use an absolute path to another place on the disk then probing isn't the solution, but I'd argue whether you needed it in another place anyway.<br />
<br />
The config change required should look something like this:<br />
<br />
<configuration><br />
<runtime><br />
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1"><br />
<probing privatePath="Modules\"/><br />
</assemblyBinding><br />
</runtime><br />
</configuration>Adrian L Thomashttp://www.blogger.com/profile/05314909861645693237noreply@blogger.com0tag:blogger.com,1999:blog-5179584950219117107.post-8733936603050907512013-07-23T13:24:00.002+01:002013-07-23T13:30:18.604+01:00Dragging a ListView control in a fullscreen WPF touch app causes the whole window to shiftI recently had a bug with the ListView control in WPF where I had a ContentControl using a ScrollViewer with the ItemsPresenter inside of it. When using touch, and navigating through the items, the whole window would shift when you reach the end of the list, which then allowed the user to access the underlying Windows shell.<br />
<br />
The cause of this is due to a potential bug in the ScrollViewer control, where the UIElement.ManipulationBoundaryFeedback event is fired, and is not handled by the ListView. As this is a routed event, it continues up the chain and eventually reaches the Window control, which then handles the event and results in an animation of the entire window.<br />
<br />
<a href="http://msdn.microsoft.com/en-us/library/ms754010.aspx" target="_blank">MSDN states</a>:<br />
<i>The ManipulationBoundaryFeedback event enables applications or components to provide visual feedback when an object hits a boundary. For example, the Window class handles the ManipulationBoundaryFeedback event to cause the window to slightly move when its edge is encountered.</i><br />
<br />
The fix (or hack) for this, is to hook in to the event and handle the event, so that it does not bubble up to the Window control.Adrian L Thomashttp://www.blogger.com/profile/05314909861645693237noreply@blogger.com0tag:blogger.com,1999:blog-5179584950219117107.post-27493218162247843302013-06-07T19:54:00.000+01:002013-06-07T19:54:55.874+01:00Telltale Games on Windows 8I recently got a pack of games by Telltale Games on the Humble Bundle Weekly Sale, however I was unable to play any of the games using Steam. The game would just begin to start and then exit. Googling didn't help much, but the support forum did. According to member Septarius: "Somewhere in the TTG engine there is a possible incompatibility with the Windows 8 version, either their fault or Microsoft's. This fault causes their games to crash when it detects a controller is attached and it tries to use it. The Windows 7 version does not have this problem."<br />
<br />
Anyway, the solution is to download dinput8.dll (can be found in the forum link) and place it into the folder of the game you want to play. So in my case, for The Walking Dead it was <i>"D:\Program Files (x86)\Steam\steamapps\common\The Walking Dead".</i><br />
<i><br /></i>
Telltale Games Support Forum: <a href="http://www.telltalegames.com/forums/showthread.php?p=682670#post682670">http://www.telltalegames.com/forums/showthread.php?p=682670#post682670</a><br />
<i><br /></i>
Although this appears to be the same for their other games, such as Hector, Wallace and Gromit, Back to the Future, and others.<br />
<br />
Hope that helps someone else having the same issue!Adrian L Thomashttp://www.blogger.com/profile/05314909861645693237noreply@blogger.com0tag:blogger.com,1999:blog-5179584950219117107.post-20274461181913819112013-04-13T18:06:00.000+01:002013-05-26T10:37:51.610+01:00Setting up xrdp for Ubuntu 12.10 (Quantal Quetzal)I decided to give Ubuntu a go on an old computer today, but I've not got a spare monitor/keyboard/mouse to hook up to it and decided I'd like to remote in to it. I could either use VNC (which actually has the same issues as xrdp anyway) or use xrdp so I can use a familiar environment like Windows Remote Desktop.<br />
<br />
However it wasn't as straight forward as installing xrdp, as I had a couple of issues, namely:<br />
<br />
<ul>
<li>Blank (plain with just the desktop) screen when connecting</li>
<li>The 'd' key caused all windows to be minimized.</li>
</ul>
<div>
<br /></div>
<h3>
Installing xrdp</h3>
<div>
This is the easy bit. Open up the terminal (Ctrl+T) and type in:</div>
<span style="font-family: Courier New, Courier, monospace;">sudo apt-get install xrdp</span><div>
<i style="background-color: #f3f3f3;"><br /></i></div>
<h3>
Fixing the plain screen</h3>
The problem seems to lay with the Unity interface, so unfortunately if you are wanting to use that then this fix isn't for you. However, you can use the older Gnome interface by first checking you have it installed:<span style="font-family: Courier New, Courier, monospace;">sudo apt-get install gnome-fallback</span><div>
<span style="background-color: white;"><br /></span></div>
and then creating an xsession file that specifies what interface to use:<div>
<span style="font-family: 'Courier New', Courier, monospace;">cd /home/adrian</span><span style="font-family: Courier New, Courier, monospace;"><br /></span><span style="font-family: 'Courier New', Courier, monospace;">echo "gnome-session --session=ubuntu-2d" > .xsession</span><span style="font-family: Courier New, Courier, monospace;"><br /></span><span style="font-family: 'Courier New', Courier, monospace;">sudo /etc/init.d/xrdp restart</span><span style="font-family: Courier New, Courier, monospace;"><br /></span><h3>
Fixing the 'd' key problem</h3>
This problem appears to be due to one of xrdp's dependencies (as I had the same issue using VNC too), and I didn't have the problem when using the machine with a keyboard directly.<br /></div>
<div>
It seems that the <super> key is ignored when remoting in, and so any key combinations that require the super key are simply ignored. Clearly this is quite frustrating and so I just disabled them.<br /><br />Open the dconf Editor:<br /><span style="font-family: Courier New, Courier, monospace;">dconf-editor</span><div>
<span style="background-color: #eeeeee;"><span style="font-family: Courier New, Courier, monospace;"><br /></span></span></div>
and navigate to:<br />org > gnome > desktop > wm > keybindings<br /><div>
<span style="background-color: white; font-family: inherit;"><br /></span></div>
Find anything with the "<super>" in it and either remove it all together (leaving behind square brackets "[]"), or replace it with another key.<br /><br />After these fixes it's all up and running great!</div>
Adrian L Thomashttp://www.blogger.com/profile/05314909861645693237noreply@blogger.com0tag:blogger.com,1999:blog-5179584950219117107.post-31176700365791554992013-02-28T19:11:00.002+00:002017-03-06T20:08:42.321+00:00Starters guide to web scraping with the HTML Agility Pack for .NETI recently wanted to get a rough average MPG for each car available on the website fuelly.com, yet unfortunately there was no API for me to access the values, so I turned to Google and came across the NuGet package <a href="http://nuget.org/packages/HtmlAgilityPack" target="_blank">HTML Agility Pack.</a> This post will get you up to speed on using HTML Agility Pack, basic XPath and some LINQ.<br />
<br />
Before we start, please make sure to check the terms and conditions and any possible copyright terms that may be applicable to the data you are retrieving. You should be able to view this on the website, however it may vary from country to country. Please also keep in mind that you will effectively be accessing the site at a rapid rate, and so it would be sensible to save any communications to your local disk for later usage, and adding a delay between page downloads.<br />
<br />
<h2>
<b>Getting Started</b></h2>
<h3>
<b>Identifying the data</b></h3>
<div>
The first thing you need to do is find where in the HTML the data is you want to download. Let's try going to fuelly and <a href="http://www.fuelly.com/car/" target="_blank">browsing all the cars</a>. As you can see there are a large variety of cars available, and if you click on one of the models it takes you to a page that displays the year of the model and the average MPG for it. For my personal project, I wanted to obtain four values: Manufacturer, Model, Year and Average MPG. With this data I can then perform queries such as what vehicles between 2003 and 2008 give an MPG figure of above 50? - However, for the purpose of this blog post and simplicity, lets simply just retrieve a list of some models from a manufacturer.</div>
<div>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfunhriwOgXnVM6ko7GTZFnMfaTMw-oAcj1FQ9lqPTG9V-iaQZFQ0FwoOpIjQ4vtZ7fqsjVkp3zjLVW04lfEnJN_SpAFs7xWEPnwuTQYNVefMA4AojkLvSpoHh66_I1IVzJIZvp18IO18/s1600/fuellybrowse.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="464" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfunhriwOgXnVM6ko7GTZFnMfaTMw-oAcj1FQ9lqPTG9V-iaQZFQ0FwoOpIjQ4vtZ7fqsjVkp3zjLVW04lfEnJN_SpAFs7xWEPnwuTQYNVefMA4AojkLvSpoHh66_I1IVzJIZvp18IO18/s640/fuellybrowse.png" width="640" /></a></div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div>
So, lets start on the <a href="http://www.fuelly.com/car/" target="_blank">browse all cars page</a>, and view the page source. If we look at the first manufacturer header on the page, we see Abarth, followed by AC, etc.</div>
<div>
<br /></div>
<div>
Open the page source (for Google Chrome: Settings > Tools > View Source), and find "Abarth":</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhoFaAhHwAN0rfKpjoA0axxbGcsxE4ZXB-2pZiGPXZZYB41CD92D_TdwH8bYcToVd8syipbXP3LqbNsuXPLcEXfs1LvnhDehBXNBEMDt67VMedgsUvNlG0W1bfc_D5BcojB5B_h5L_rFuc/s1600/fuellysource.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="484" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhoFaAhHwAN0rfKpjoA0axxbGcsxE4ZXB-2pZiGPXZZYB41CD92D_TdwH8bYcToVd8syipbXP3LqbNsuXPLcEXfs1LvnhDehBXNBEMDt67VMedgsUvNlG0W1bfc_D5BcojB5B_h5L_rFuc/s640/fuellysource.png" width="640" /></a></div>
<div>
<br /></div>
<div>
You will see that the Manufacturer is wrapped in a <h3> tag. Below this, is a <div> that contains each model under the Abarth name: 500, Grande Punto and Punto Evo. Let's try and get hold of these models, but first, we need to setup the project.</div>
<div>
<br /></div>
<h3>
<b>Setting up the project</b></h3>
<div>
Create a new project in Visual Studio, a simple console project should suffice for this blog post. Add the HTML Agility Pack to the project via NuGet and add the following code:</div>
<pre style="background-image: URL(https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhImMCYf0zWnOzlfyFoNQEB8c5vssDy7UUISylxbnCkYLdpQVQNP9IWjNW4UQgQDgFAzCT1lCXKxJXbOfs0pyazgkn2bAxVEf8EoTu0TsaJcv0XUXXrv0tO8g8pMV39f92S_qZOGX93-WFv/s320/codebg.gif); background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">1: class Program
2: {
3: static void Main(string[] args)
4: {
5: const string WEBSITE_LOCATION = @"http://www.fuelly.com/car/";
6: var htmlDocument = new HtmlAgilityPack.HtmlDocument();
7: using (var webClient = new System.Net.WebClient())
8: {
9: using (var stream = webClient.OpenRead(WEBSITE_LOCATION))
10: {
11: htmlDocument.Load(stream);
12: }
13: }
14: }
15: }
</code></pre>
<div>
<br /></div>
<div>
This simply loads the HTML page in to the HtmlDocument type so that we can run XPath queries against it to eventually get the value we are looking for.</div>
<div>
<br /></div>
<h3>
Navigating Nodes with XPath</h3>
<div>
So, as noted earlier, we want to get a load of models from a manufacturer. Each <div> tag has a specific ID which we can use in our query (see "inline-list" below).<br />
<pre style="background-image: URL(https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhImMCYf0zWnOzlfyFoNQEB8c5vssDy7UUISylxbnCkYLdpQVQNP9IWjNW4UQgQDgFAzCT1lCXKxJXbOfs0pyazgkn2bAxVEf8EoTu0TsaJcv0XUXXrv0tO8g8pMV39f92S_qZOGX93-WFv/s320/codebg.gif); background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">1: <h3><a href="/car/abarth" style="text-decoration:none;color:#000;">Abarth</a></h3>
2: <div id="inline-list">
3: <ul>
4: <li><nobr><a href="/car/abarth/500">500</a> <span class="smallcopy">(34)</span> &nbsp; </nobr></li>
5: <li><nobr><a href="/car/abarth/grande punto">Grande Punto</a> <span class="smallcopy">(1)</span> &nbsp;</nobr></li>
6: <li><nobr><a href="/car/abarth/punto evo">Punto EVO</a> <span class="smallcopy">(3)</span> &nbsp; </nobr></li>
7: </ul>
8: </div>
</code></pre>
<br />
We can use this specific hook to get the values we want. So lets add the code:<br />
<pre style="background-image: URL(https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhImMCYf0zWnOzlfyFoNQEB8c5vssDy7UUISylxbnCkYLdpQVQNP9IWjNW4UQgQDgFAzCT1lCXKxJXbOfs0pyazgkn2bAxVEf8EoTu0TsaJcv0XUXXrv0tO8g8pMV39f92S_qZOGX93-WFv/s320/codebg.gif); background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">1: HtmlAgilityPack.HtmlNodeCollection divTags = htmlDocument.DocumentNode.SelectNodes("//div[@id='inline-list']");
</code></pre>
<br />
<span style="font-family: "verdana" , "helvetica" , "arial" , sans-serif; font-size: 12px; line-height: 16px;">The above code returns a collection of <div> tags that represent each Manufacturer listed on the page. </span><br />
<span style="font-family: "verdana" , "helvetica" , "arial" , sans-serif; font-size: 12px; line-height: 16px;"><br /></span>
Let's break up <a href="http://www.w3schools.com/xpath/xpath_syntax.asp" target="_blank">the XPath syntax</a> to make sense of it:<br />
<span style="font-family: inherit;">// - </span><span style="font-family: "verdana" , "helvetica" , "arial" , sans-serif;"><span style="font-size: 12px; line-height: 16px;">Selects nodes in the document from the current node that match the selection no matter where they are.</span></span><br />
<span style="font-family: "verdana" , "helvetica" , "arial" , sans-serif;"><span style="font-size: 12px; line-height: 16px;">div - The specific nodes we are interested in.</span></span><br />
<span style="font-family: "verdana" , "helvetica" , "arial" , sans-serif;"><span style="font-size: 12px; line-height: 16px;">[@id] - Predicate that defines a specific node.</span></span><br />
<span style="font-family: "verdana" , "helvetica" , "arial" , sans-serif;"><span style="font-size: 12px; line-height: 16px;">[@id='inline-list'] - Predicate that defines a specific node with a specific value.</span></span><br />
<span style="font-family: "verdana" , "helvetica" , "arial" , sans-serif;"><span style="font-size: 12px; line-height: 16px;"><br /></span></span><span style="font-family: "verdana" , "helvetica" , "arial" , sans-serif;"><span style="font-size: 12px; line-height: 16px;">We can now dig deeper into each div tag, using a little more XPath and some LINQ to get the values we want.</span></span><br />
<span style="font-family: "verdana" , "helvetica" , "arial" , sans-serif;"><span style="font-size: 12px; line-height: 16px;"><br /></span></span></div>
<h3>
Accessing the data using LINQ</h3>
<div>
<span style="font-family: inherit;">OK, so within each item of the <div> tags we still have a load of rubbish we don't really need. All we want is the car models. Well we know each car model is between <a> (hyperlink) tags, with a href value. So using XPath and a little LINQ we can extract the data we need:</span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<pre style="background-image: URL(https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhImMCYf0zWnOzlfyFoNQEB8c5vssDy7UUISylxbnCkYLdpQVQNP9IWjNW4UQgQDgFAzCT1lCXKxJXbOfs0pyazgkn2bAxVEf8EoTu0TsaJcv0XUXXrv0tO8g8pMV39f92S_qZOGX93-WFv/s320/codebg.gif); background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">1: HtmlAgilityPack.HtmlNode aTags = divTags.FirstOrDefault();
2: var manufacturerList = from hyperlink in aTags.SelectNodes(".//a[@href]")
3: where hyperlink != null
4: select hyperlink.InnerText;
</code></pre>
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Line 1: Get the first manufacturer in the list </span><br />
<span style="font-family: inherit;">Line 2: for each hyperlink inside the div, select all <a> tags with a 'href' node</span><br />
<span style="font-family: inherit;">Line 3: where the hyperlink isn't null (i.e. a href node was found), then:</span><br />
<span style="font-family: inherit;">Line 4: select the text inside of the hyperlink.</span><br />
<span style="font-family: inherit;"><br /></span></div>
<h3>
Conclusion</h3>
<div>
So, there you have it. You learned how to get specific values within a piece of HTML, a little XPath and some LINQ. Although the end data in this example probably isn't too useful, hopefully you can now see how you would expand upon this to find specific values and URLs to build a more complex system.</div>
<div>
<br /></div>
<div>
<a href="https://github.com/AdrianLThomas/Blog/blob/master/Assets/Blogger/2013/Code/WebScrapingWithHtmlAgilityPackSource.zip?raw=true" target="_blank">You can download the source here.</a></div>
<div>
<br /></div>
Adrian L Thomashttp://www.blogger.com/profile/05314909861645693237noreply@blogger.com9