It is amazing to me the relationship that (almost) every jeep owner has towards another jeep owner. Whether I know the jeep driving towards me or not I feel compelled to wave at them as they drive by. Generally they reciprocate with a head nod or a wave back. If we stop next to one another we immediately check out the other fella’s ride. If the windows are down we exchange compliments.
I have worked at Clear Measure for a few weeks now. We moved into our new building about a week ago. And with my new window view I see a yellow jeep and a blue jeep drive in and out several times through out the day. I finally had a chance to go introduce myself as a fellow jeep owner. We looked at his yellow jeep. And talked about future mods.
Then we walked into the garage and looked at my jeep and Jeffrey’s jeep (parked side by side so that they can have some alone time throughout the day – if we are lucky they will make more white Rubicon JK’s!). This then led to “what do you do”. We are both C# developers. This then led to a tour of his office on the 5th floor in the same building as Clear Measure. Then a business card exchange. Ending with a “we should meet on the trail one of these days”.
The only other community that I have come even close to this level of immediate friendship is in the “back to the land” or sustainable farming community. The difference being that we really expect helping hands and knowledge exchange from one another. Where as jeepers generally expect only a wave or a tow out of a sticky situation.
I acknowledge that there is a similar friendliness in the motorcycle group…but there you often have harley snobs, rice rocket racers, and leather gangs.
I love being a jeep owner!
My name is Andrew Siemer. I recently left Dell and started working with Jeffrey Palermo and the gang over at Clear Measure. As soon as I heard that there was an opportunity to get this Azure group up and moving again I jumped on it. I very much look forward to serving you all!
Please let me know if you find any interesting content that would appreciated by this group in a weekly newsletter. Even better - if you make that content send it my way! Also, if you are interested in presenting feel free to reach out to me and I will get you scheduled. Would your company like to sponsor the user group? All we need is pizza and sugar water in exchange for a few minutes with a captive audience!
Also, we are planning on moving this from a virtual event to an in person event. We are targeting the Microsoft campus - 10900 Stonelake Blvd, Austin. And we are looking to get this scheduled for every third wednesday of the month at 6pm.
Azure info for the week:
- Azure Automation Capabilities in Depth: The Azure Automation PowerShell Cmdlets
- Azure Virtual Machine Disk Encryption using CloudLink
- MyGet Now Available in the Azure Store
- Enabling Command-line or Continuous Delivery of Azure WebJobs
- SQL Server High Availability in Windows Azure Iaas
- Updates Azure PowerShell directly from GitHub
- Landlord for server environment automation
- Azure Exam: 70-533 Implementing Microsoft Azure Infrastructure Solutions
- Building Cloud Apps with Microsoft Azure
- Microsoft Azure courses
- If you like building cutting edge apps targeted for Auzre come apply at the coolest engineering shop in Austin - Clear Measure! We are always looking for talented people. http://www.clear-measure.com/careers/
I have now been at Clear Measure for a week. Today is actually the end of my first week. I have to say that I love it! I have worked for many consulting type companies before so I have quite a few to compare too.
Like any mostly new company that is just getting itself started – the most important factor is the people that make up the team. This team is a great one. I have not met a single person that isn’t here to help every other person. This is hugely important.
Let’s quickly get the standard Joel Spolsky test out of the way. We get a score of 8 out of 12. We have two NO’s. One maybe and one sort of. Not too shabby!
- YES - Do you use source control?
For client projects we use Visual Studio Online (TFS) with GIT. For open source projects we use GitHub.
- YES - Can you make a build in one step?
It is our goal to always have CI for every project. Generally this comes first. Sometimes it comes a few steps into the project. But this is very high on our list of importance. TeamCity is the preferred tool for this job.
- YES - Do you make daily builds?
This again requires that the CI for the project is in place but once it is daily builds if not more often are always the goal.
- YES - Do you have a bug database?
We use VSO for some of this, Trello for others.
- maybe - Do you fix bugs before writing new code?
In a customer driven environment, this is ultimately up to the customer. But as software craftsmanship is our core competency we do plead our case as best we can about fixing bugs. Sadly, we don’t always win this fight and from time to time bugs in the system are not prioritized over new features at all times.
- YES - Do you have an up-to-date schedule?
Our VP’s of Engineering…the guys who run each of our projects – know what they are doing as it pertains to the customer, the project, the code, and the schedule. There are very few non-technical folks in the company and that’s great!
- YES - Do you have a spec?
- no - Do programmers have quiet working conditions?
As the company is young it had to start small in terms of office space. Everybody from the CEO to the interns are sitting in shared space. We are moving to a new office at the end of the month where I am told the quite developer folks will be separated from the loud business/sales folks. But I believe we will still be
- YES - Do you use the best tools money can buy?
Yes! Always. I have yet to hit a wall when asking for a tool to do my job or make the company a better place.
- sort of - Do you have testers?
Everyone is a tester don’t ya know? Have not yet met a full time real QA person. But that isn’t to say that we don’t have people responsible for testing. We have several folks that start in the world of SDET.
- YES - Do new candidates write code during their interview?
Our interview does indeed require you to write code. At first pass I thought “how simple was that”. But after being on the other side listening to calls and hearing about the test results of various folks, I am shocked to learn how such a simple test is able to weed out so many folks!
- no - Do you do hallway usability testing?
…but to be fair we don’t yet have any hallways! :)
If you haven’t seen this test before take a look here. It’s priceless for us developery folk! http://www.joelonsoftware.com/articles/fog0000000043.html
Although its only been a week…I am looking forward to my future here. Come join me!
This week was Logan’s 11th birthday. Some of you may know that he and I are really enjoying competing in three gun together. But as he is left handed, and still young at a fresh 11, shooting some of my gear is difficult for him either due to being right hand specific or heavy for a little guy. To alleviate some of those issues I had a custom left handed light weight rifle built for him. My rifle is 6 pounds and right handed. His is 4 pounds and left handed.
A neighbor of ours has sold his house. He is basically done with the life of having property. Well, the work of having property. He decided to leave Texas and head back to Florida for walks on the beach. As he no longer needed his stuff he decided to have an everything must go sale. A buddy and I went in on a convertible trailer. Part flat bed. Part stock trailer. Perfect for small time ranchers like us who need a flat bed near daily and a stock trailer now and then.
The first major task for us was to get some hay for the horses. This meant that we needed to take the top off. Uh oh…no where to put it! I know – hang it from the barn rafters (hope that old barn holds). Backed it in. Hung it up. Ratcheted it off.
And we will NEVER do that again! Poor old barn didn’t know what we were doing to it!
Then off we went to get some hay.
Four bales at a time is way better than two!
The horses didn’t know what it was that we were doing unloading huge piles of food at night. But they couldn’t wait to get into it.
And the kids couldn’t help but play in the hay as well.
Everyone at one point or another in their career fantasizes about how cool it would be to work from home. They always joke about taking meetings in their under wear. Do a little work with your family in the other room. Have lunch with the family. Be as comfy as you want to be. And be able to pay the bills with that life style.
Coding challenge number two has been thrown down. The nice thing about having two competitive kids learning something together is that they inevitably push each other to do more than the teacher (me) asked. They have been hard at work figuring out the first challenge and were immediately ready for the next thing.
So…I hollar’d “C H A L - L AAAA N G E” !!!!! $5 to the winner (first to complete)
This exercise is to extend the concept of a single document to a “web”. Three documents and two images. All linked together correctly. As before, a lo-fi mock has been provided.
Hard at work helping each other out already.
I foresee lots of phone calls at work tomorrow.
I am teaching some of my kids how to write html and dabble in css. This has been quite a bit of fun. More than I thought it would be actually. In fact, as soon as I was done giving my older boy his first lesson, one of my girls immediately wanted lessons (they are very competitive). Now I have two learning how to write html. I love this.
During the first lesson I ask them to follow a simple wire frame which shows how to add a header, an image, some wrapped text, a table, and some simple formatting. This meant that we had to go find a picture. In doing so the kids had a chance to see a long ugly Facebook url.
Of course anyone not in the know would see that and say YUCK! In explaining loosely what it is and how it works I came up with this scenario:
The http part is how you call me from down the hall and say "hey dad”. The other part of the long string is the rest of the message “can you give me my green shirt?" to which I toss you your green shirt.
Where as the “s” in https is a more secure way to do the same thing. Only in this case you call down the hall and say "hey dad can you give me my green shirt...the one with my bank account # written on the front"...but before I toss the shirt to you I take the shirt apart and instead toss you a ball of thread with instructions on how to put it back together again.
This way your attackers have a harder time figuring out the important information – the bank account number. And as with anything where I want to make it harder for someone else to read...it takes a little bit longer for me to read it too.
I didn’t cover the various scenario’s where the kid screams down the hall asking me for his green shirt to which I reply
- “I can’t find it” – 404
- “Your door is locked” – 401
- When there is more than one green shirt – 300
- When the requested shirt is dirty and in the laundry - 307
- I am busy practicing making another kid with your mother – 408
- When the daughter asks for her short shorts and tight top – 403 or 406
- Or in the near future when you kid thankfully no longer lives with you – 301
More of these analogies can be found here: http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
As a full time web developer that has been paying the bills for a great many years with self taught web skills, I finally decided that my kids were ready for me to pass the baton. It is my hope that my kids will go to college and learn what ever tickles their fancy. But no flipping burgers for my monsters. Instead I fully intend to enter them into the world wide job market that is the inter-webs ASAP. They can easily afford college with simple programming skills. How better to start them off than teach them first and foremost how to learn by speaking Google. This is a very important distinction over actually teaching HTML. Learning how to look for information, absorb only what is needed for the immediate task at hand, as well as get a cursory understanding of all the peripheral information is how I work every day. Rather than teach this tag or that tag I gave him a simple understanding of how to feed in the proper english term set to Google to get the results you need. Lets do this!
As with any web developer job, I started Drake off with a low-fidelity wire frame. I asked him to create a basic web page that has a header, his picture, some paragraphs of text with the first paragraph wrapped around his photo. 3 paragraphs about himself. He asked me if it was to be in “first person” or not. I love this boy. And at the bottom I asked for a simple table with his personal information.
He immediately went to the googles and found w3schools. The best place for the low down on all html tags.
He doesn’t know it yet, but this is one of the few gigs in life where you can easily make well over 6 figures in your bath robe from your home every day.
Getting this boy to sit style while smiling is not easy.
Here you can see that he got his image to render from Facebook. Now he is smiling again.
As a farmer there are many things you need to do on a regular basis. Similar to being a software developer, you have to check your email at some interval no matter how much you like it. Well, as a farmer I have to walk my fence line with some frequency. And like email, now and then you come across fun surprises with the other lame task.
While walking my fence line, both internal fences (where I keep my pigs), and external fences (which keep out the neighbors stock), I ran across a new batch of cattle in the back acreage. Curious critters. They wanted to know as much about me as I did them.
Its very easy for me to see why there are so many lawyers, doctors, and engineers, (the top three professions dropping out of city life at an alarming rate) returning to the farming life style. All that stress in the job contrasted with walking along and seeing gentle cows chewing their cud. Farming, while always busy, is generally very low stress and enjoyable.
How many folks have a parks worth of grass in their front lawn to toss the football with…with your kids, dogs, horses, and cats?
I love this life.
My dad recently passed away. This has left me thinking quite a bit about my relationships with all of my kids (I have six if you didn’t know). And while I didn’t really know all that much about my father, I am keenly aware of the things that we did and didn’t do together. I think I like most of the things that I remember about my dad and how he raised me and I see that I have replicated much of this in my relationships with my kids. However, I also see me doing many of the things that I didn’t really enjoy. I am now actively trying to change those items.
It is odd to me how important the simple act of throwing a ball with your father is to a boy. I never really thought much about this. But these days I think about it a lot. I never really had the opportunity to do this with my old man as he never really had time…and was sort of old. I know now that having time and making time are just different ways to manage the many things that are going on in life. My boys love throwing the ball. And I really enjoy it too. And it really only takes a few minutes a day to check a box of this nature. And the memories that are built from it set the foundation for many other things in life.
From a home schooling perspective, throwing the foot ball is so much more than the bond made between father and son. Hand eye co-ordination. Tactics. Judgment. Gamesmanship. All important topics that are difficult to teach in other formats regularly taught at home. Add to that three flies up with dog piles and brotherly punishment and you end up with tough happy monsters…with good memories of a certain boyhood requirement.
In order to meet some of my goals at work I decided to contribute my thoughts around Html Helpers and AngularJs support.
Here is my first nuget package: https://www.nuget.org/packages/AngularJs.Mvc
This package provides two things.
1. HtmlHelpers that output angular friendly input controls
2. FluentValidation integration to provide a better client/server validation story similar to the jquery unobtrusive concepts
This package is pretty young and has quite a bit of work remaining. But it is also somewhat useful in its current state.
Take a look!
As you may know I recently started a new job at my old company. As the new guy I have been sitting in as many meetings as possible trying to learn as much as I can about the underlying system, our methodology, what is important to the business, how we are going to achieve our deadlines, etc. During these meetings I have heard mention of the “ShortBus” time and time again. This was odd as we are building a system around NServiceBus…an enterprise level message bus (among other things). I was really under the impression that there was some great divide between the engineers “THEM” and the architects “ME” (…and my team). I really thought they were dogging us! Gonna have to push a boulder up hill in order to move a pebble…here we go again.
I finally asked today…what is this ShortBus you keep talking about?
It turns out that a a co-worker of mine, Matt Hinze, created a little toolset that is sort of a pub / sub in memory bus that also seems to tie in well with some CQRS-ish patterns. They use it primarily as a mediator between reading and writing data. It is perfect if you want to decouple your system a touch more.
In my case I am looking to remove a skosh of complexity in order to make my tiny CMS that much more testable but also to make it more extensible. I haven’t really posted anything about my CMS. But…my CMS is non-standard, built on top of AWS, and using the AWS blob storage (S3) almost entirely as its data store. This was done primarily as a spelunking exercise in the Amazon eco-system. It was also done so that I could get rid of my big metal hosting provider. I went from $250/mo down to $10 or so.
Given the store I ended up having a series of indexes to keep up to date which allows me navigate the structure in various ways. This works great for me. But managing a denormalized read layer as I throw new bits at it would be much easier if I published an event to the system and had a bunch of workers that just responded to that. It would mean a new feature would simply be a new set of event handlers.
Let’s give it a try…
Thank you Matt Hinze!
Today I started a new gig in my old company Dell. I am joining the edell transactional team as one of the architects. I am experiencing some of the slowness that comes with a big company like this. However I am really enjoying the conversations I have been a part of so far and am really enjoying all the energetic conversations that the engineers around me are having.
I am very happy I chose to stay with Dell. It looks like this could be more fun than most people think Dell is. More to come soon.
I have to go brush up on nservicebus, angular, bootstrap, gridsetapp, and sherpa.
I am trying my best to get back into blogging. After having moved to Texas and starting my own ranch/farm…I have had difficulties staying focused on technology when I am not work. For that reason I will try to do quick posts while at work…while I am focused on technology.
To start this quick story I have to first mention that we have released several iterations of our product “Marketvine Picks” where Marketvine is the company we are spinning out of Dell as and Picks is the social commerce-y product we have created. Currently you can see this product on www.citysports.com, www.eyeslipsface.com, and as of last night on www.surlatable.com.
Using read replicas from an infrastructure point of view is not always easy. However, we are in AWS…so this feature is as easy as adding a new read replica. Right click on an RDS instance and add a read replica. Quick modal configuration. GO…minutes later we have a read replica.
Depending on how you chose to structure your application you may or may not be ready to utilize this easy to use feature for horizontal scaling. Going wide with your database if you will…rather than standing up a more powerful database server. You can have up to 5 read replicas per each master instance. NICE!
In our case we made an decision early on to at least contain our read queries and our write queries in separate containers. We pretty successfully conform to the concept of command query separation (CQS). This allowed us to easily tackle the concept of a read connection string and a write connection string later on.
Well…later on is here now!
We now have two separate connection handlers 1) MySQLReadonlyDatabaseFactory and 2) MySQLReadWriteDatabaseFactory. But how to get this new concept into our code? Thank god for IoC (Ninject in our case).
We have marker interfaces on all the classes that do reads vs. writes. If the class does solely reads they are marked with IQueries. If they do solely writes then they are marked with ICommander. This has then allowed us to swap in our new functionality with just a few lines of IoC configuration code:
This is a repost from here in case I need it again: http://www.codeproject.com/Articles/539088/Amazon-AWSDeploy-To-Provision-Multiple-Websites
Recently I've been on a project where we are implementing BlueGreen deployments using Amazon CloudFormation, and came across a number of unanswered questions:
- How do you deploy multiple applications to a single instance and support auto-scaling?
- How do you deploy non-website applications using AWSDeploy?
- How can you pass in environmental config during provisioning?
The official answer to the first 2 questions appears to be that you can't without additional scripts/tools. The third (with some caveats) is that you need to bake them into your package prior to your deployment.
This seems frustrating for such common real world use cases, but luckily there is a solution which requires minimal effort.
First Some Background
CloudFormation is Amazon's offering for provisioning a collection of AWS resources on the amazon infrastructure through declarative templates. The collection of resources is known as a stack and can be used for reliable and repeatable infrastructure deployments.
AWSDeploy is the tool which can be used with the Microsoft stack to provision CloudFormation templates and to deploy web applications onto these provisioned resources. This uses Microsoft WebDeploy under the hood, and as such a WebDeploy package must be provided when provisioning a CloudFormation stack using the tool.
The WebDeploy package provided by the user is uploaded to AWS by the tool and stored in an S3 bucket. When the EC2 web server spins up, AWSDeploy installs the package on the web server as part of initialising the stack. By default any failures will result in a stack rollback.
Once a web application is provisioned via AWSDeploy, the stack will manage failover and auto-scaling scenarios automatically by redeploying the associated WebDepoy package stored on S3 to any new instances. In doing so it is able to manage resources hands-free and replicate perfect copies of your application on demand.
With Great Power Comes Great Limitations
As described above, AWSDeploy is incredibly useful for failover and scaling scenarios. Your web application can be replicated and auto-provisioned without manual interaction.
The downside is a number of limitations:
- Only 1 WebDeploy package can be associated with a web server instance
- No parameters can be passed when AWSDeploy installs the WebDeploy package
- As such WebDeploy parameterisation cannot be used for config transforms
- Environmental config is limited to a small set of pre-defined appSettings keys, which can be injected during provisioning
The impact is that to work with AWSDeploy you generally have to modify your build artifact for every deployment environment and bake in configuration settings. This is against Continuous Delivery principles and adds complexity to any deployment scripts.
WebDeploy in it's own right is an extremely flexible and versatile tool. In a single WebDeploy package it's possible to deploy multiple websites, deploy directories of arbitrary files, execute commands, and change Acl permissions.
So, instead of baking environment configuration into the WebDeploy packages to support AWSDeploy, why not instead create a wrapper WebDeploy package and use an embedded
RunCommand to pass environment variables to a child package during provisioning. In this way the original build artifact is unmodified and the full power of WebDeploy can be used to inject parameters during installation.
This approach has additional benefits in that it allows the install of multiple web applications via a single AWSDeploy package, as well as the ability to deploy Windows Services and Console applications during auto-provisioning. Additionally it's then relatively easy to cheery-pick applications for deployment to a particular instance with very little rework of deployment scripts.
On creating my first wrapper package I immediately ran into a problem when provisioning. The AWSDeploy logs revealed the issue.
Collapse | Copy Code
ERROR 10 AWSDeploymentHostManager.Tasks.UpdateAppVersionTask - Deploy failed, not iisApp.
It seems AWSDeploy expects the WebDeploy package to contain an
iisApp within it's manifest. So I included a stub website in my WebDeploy package and tried again. Surprisingly I hit the same error!
After much investigation it turns out that when AWSDeploy executes the WebDeploy package, it passes the following parameter to the package. This is to enable customisation of the target IIS Website name during provisioning, and can be set via a AWSDeploy configuration setting.
Collapse | Copy Code
-setParam:"IIS Web Application Name"="Default Web Site/"
Knowing the parameter requirement, the solution is simple. Provided the WebDeploy package contains a parameters file with this parameter defined within, then AWSDeploy will successfully deploy the package regardless of it's contents. It also means that the WebDeploy package does not actually need to include an
iisApp instance as the error first suggested.
Putting It All Together
First place any WebDeploy packages to be installed on the server instance in a folder called
C:\Packages. Include any set-parameter configuration files for any environmental settings and config transforms.
Create a PowerShell install script called
Install.ps1 and add this to the
C:\Packages directory. The script will be executed on the server during provisioning. It will be used to deploy the WebDeploy packages in this folder on the server, and can be used to invoke explicit WebDeploy commands during install; such as to leverage environment specific parameterisation.
Collapse | Copy Code
Set-Alias msdeploy "C:\Program Files\IIS\Microsoft Web Deploy V3\msdpeloy.exe" msdeploy -verb:sync -source:package=WebsiteA.zip -dest:auto -setParamFile:WebSiteA-Params.xml msdeploy -verb:sync -source:package=WebsiteB.zip -dest:auto -setParamFile:WebSiteB-Params.xml
Next create the parameters file for inclusion in the WebDeploy wrapper package as follows. Name the file
parameters.xml and place it in a folder called
Collapse | Copy Code
<parameters> <parameter name="IIS Web Application Name" defaultValue="Default Web Site/" tags="IisApp" /> </parameters>
Now create a WebDeploy manifest file for the wrapper package as follows. This will provision the contents of
C:\Packages to the server instance and use the
RunCommand provider to execute the
Install.ps1 PowerShell script in the root of this directory. Name the file
manifest.xml and place it in the
Collapse | Copy Code
<siteManifest> <contentPath path="C:\Packages" /> <runCommand path="%SystemRoot%\System32\WindowsPowerShell\v1.0\PowerShell.exe -inputformat none -command "Set-ExecutionPolicy Unrestricted -force"" waitInterval="10000" /> <runCommand path="%SystemRoot%\System32\WindowsPowerShell\v1.0\PowerShell.exe -inputformat none -command C:\Packages\Install.ps1" waitInterval="300000" /> </siteManifest>
Finally package up the WebDeploy wrapper package using the command line as follows, which should be executed from the
C:\Build folder. The trick is the
declareParamFile parameter and
parameters.xml file which keeps AWSDeploy happy.
Collapse | Copy Code
msdeploy -verb:sync -source:manifest=manifest.xml -dest:package=Deploy.zip -declareParamFile=parameters.xml
The WebDeploy package produced is now ready to be used with the AWSDeploy tool to provision 1 or more applications on your EC2 server. It is also fully compatible with auto-scaling and failover scenarios.
You can test the install locally by executing the following command in the
Collapse | Copy Code
msdeploy -verb:sync -source:package=Deploy.zip -dest:auto
The solution minimises the customisation of deployment scripts for different environments, and results in a solution much less coupled to Amazon AWSDeploy. It also works around the limitations of AWSDeploy and provides a much greater level of flexibility in how instances can be configured to support auto-provisioning.
This is an improvement, but we are still having to build different artifacts per environment which is not ideal. A further enhancement would be to leverage AWS
UserData from within the PowerShell script.
UserData is configured during provisioning and made available to each server instance via a web service at a fixed address:
Collapse | Copy Code
UserData is persisted during failover & scaling, and could be used to either feed the PowerShell script entire set-parameter files dynamically, or provide flags to indicate to the PowerShell script which pre-deployed set-parameter file to use per environment when installing packages. In this way a single artifact could be used across all environments inline with best practice, as well as a single install and deployment script.