Generate a list of objects – Part 2

In my previous post I showed a c# Linq extension method for LINQPad that will create a list of objects from your database as C# code that you can paste in visual studio so you don’t have to manually new up an object and initialize its numerous properties.  There was one limitation in the last edition and that was it didn’t work with IQueryable<T> and also it wasn’t as seamless as you are used to writing linq queries. For example, I had to specify the name of the object “Person” as shown in the code below in the GetClist(“Person”) function.  I wanted to make it better. 

var people = from p in Persons
select new
{
   p.BusinessEntityID,
 p.PersonType,
   p.NameStyle,
    p.Title,
    p.FirstName,
    p.MiddleName,
   p.LastName,
 p.Suffix,
   p.EmailPromotion,
   p.AdditionalContactInfo,
    p.Demographics,
 p.Rowguid,
  p.ModifiedDate
};
people.Take(2).ToList().GetClist("Person");

Imagine you had bunch of orders and you wanted to print out C# code listing orders and related customers. In my latest attempt it works like below code listing. Much cleaner and concise that way you are used to writing lamda expressions.

Orders.Take(2)
.ToClist(x => x.Customer)
.ToClist();

Notice: GetClist() function is renamed to ToClist().

new List() {
new Orders() {
OrderID = 10248,
CustomerID = "VINET",
EmployeeID = 5,
OrderDate = Convert.ToDateTime("7/4/1996 12:00:00 AM"),
RequiredDate = Convert.ToDateTime("8/1/1996 12:00:00 AM"),
ShippedDate = Convert.ToDateTime("7/16/1996 12:00:00 AM"),
ShipVia = 3,
Freight = 32.3800,
ShipName = "Vins et alcools Chevalier",
ShipAddress = "59 rue de l'Abbaye",
ShipCity = "Reims",
SHIPREGION = null ,
ShipPostalCode = "51100",
ShipCountry = "France"
},
new Orders() {
OrderID = 10249,
CustomerID = "TOMSP",
EmployeeID = 6,
OrderDate = Convert.ToDateTime("7/5/1996 12:00:00 AM"),
RequiredDate = Convert.ToDateTime("8/16/1996 12:00:00 AM"),
ShippedDate = Convert.ToDateTime("7/10/1996 12:00:00 AM"),
ShipVia = 1,
Freight = 11.6100,
ShipName = "Toms Spezialitäten",
ShipAddress = "Luisenstr. 48",
ShipCity = "Münster",
SHIPREGION = null ,
ShipPostalCode = "44087",
ShipCountry = "Germany"
}
};
-----------
new List() {
new Customers() {
CustomerID = "VINET",
CompanyName = "Vins et alcools Chevalier",
ContactName = "Paul Henriot",
ContactTitle = "Accounting Manager",
Address = "59 rue de l'Abbaye",
City = "Reims",
REGION = null ,
PostalCode = "51100",
Country = "France",
Phone = "26.47.15.10",
Fax = "26.47.15.11"
},
new Customers() {
CustomerID = "TOMSP",
CompanyName = "Toms Spezialitäten",
ContactName = "Karin Josephs",
ContactTitle = "Marketing Manager",
Address = "Luisenstr. 48",
City = "Münster",
REGION = null ,
PostalCode = "44087",
Country = "Germany",
Phone = "0251-031259",
Fax = "0251-035695"
}
};

With the above code, you can show first two orders and also who placed them. Lets take it one level deep.  In the next query, you can find out orderdetails and find its related orders and take only distinct orders and list customers who placed those orders as C# code.

OrderDetails.Take(5)
.ToClist(x => x.Order).DistinctBy(z => z.OrderID)
.ToClist(y => y.Customer)
.ToClist();

For simplicity, I am not showing results of the query.  Now the code that makes this happen.

//ToClist --- on IQueryable.
 public static IQueryable ToClist(this IQueryable query, Expression> predicate)
 {
  var result = predicate.Compile(); 
  var tresult = from x in query.ToClist()
       select result.Invoke(x); 
  return tresult;
 }
 //ToClist --- on IEnumerable.
 public static IQueryable ToClist(this IEnumerable query, Expression> predicate)
 { 
  var result = predicate.Compile(); 
  var tresult = from x in query.AsQueryable().ToClist()
       select result.Invoke(x);
  return tresult;
 }
 public static IQueryable ToClist(this IQueryable value)
   {   
     Type typeName = value.GetType();  
  Type typeList = value.GetType().GetGenericArguments()[0];          
  
  var listBuilder = new StringBuilder();
      listBuilder.AppendLine(String.Format("new List<{0}>() {{", typeList.Name));                
        foreach (var element in value.ToList())
      {
   
            Type type = element.GetType();
   
   var memberInfo = type.GetMembers();
   var fields = type.GetFields();
   
   var field = fields[0].Name;
   
   var val = fields[0].GetValue(element);
   
   var properties = type.GetProperties(BindingFlags.Public);   
   
   var classBuilder = new StringBuilder();
          classBuilder.AppendLine(String.Format(@"new {0}() {{", typeList.Name));
          List lstElements = new List();
           foreach (FieldInfo p in type.GetFields())
            {
               if (p.GetValue(element) != null)
                {               
                    if (p.GetValue(element).GetType().Name =="DateTime")
                    {
                       lstElements.Add(String.Format(@"{0} = Convert.ToDateTime(""{1}"")", p.Name, p.GetValue(element)));
                  }else if(p.GetValue(element).GetType().Name == "String"){
                       lstElements.Add(String.Format(@"{0} = ""{1}""", p.Name, p.GetValue(element))); 
                 }else if(p.GetValue(element).GetType().Name == "Decimal"){
                      lstElements.Add(String.Format(@"{0} = {1}",p.Name, p.GetValue(element))); 
                  }else if(p.GetValue(element).GetType().Name == "Int32"){
                        lstElements.Add(String.Format(@"{0} = {1}",p.Name, p.GetValue(element))); 
    }else if(p.GetValue(element).GetType().Name == "Int64"){
                        lstElements.Add(String.Format(@"{0} = {1}",p.Name, p.GetValue(element))); 
                  }else if(p.GetValue(element).GetType().Name == "Boolean"){
                      lstElements.Add(String.Format(@"{0} = {1}",p.Name, p.GetValue(element))); 
                  }else if(p.GetValue(element).GetType().Name == "Guid"){
                     lstElements.Add(String.Format(@"{0} = new Guid(""{1}"")",p.Name, p.GetValue(element))); 
                    }else{      
                    }               
                } else{     
         lstElements.Add(String.Format(@"{0} = null ",p.Name.ToUpper())); 
    }
            }           
            for(int i = 0; i < lstElements.Count;i++){
               if (i == lstElements.Count - 1)
             {
                   classBuilder.Append(String.Format("{0}",(lstElements.ToList()[i].ToString())));
             }else{
                  classBuilder.AppendLine(String.Format("{0},",(lstElements.ToList()[i].ToString())));
                }
           }       
            listBuilder.AppendLine(classBuilder.ToString());
            listBuilder.AppendLine(@"},");      
        }
       var preFinal = listBuilder.ToString().Remove(listBuilder.ToString().LastIndexOf(','), 1);               
        var finalValue = String.Format(@"{0}}};", preFinal);
        Console.WriteLine(finalValue);
  return value;
   }

Let me know if you have any feedback in the comments below.

Generate List of objects for easier Unit Testing using LINQPad Extension method

Problem: When you are doing unit testing you are often faced with testing against some mock repository or some object store. The actual repository could be SQL repository or Azure repository or even CSV files . Inside a unit test you want some fake data and that data is easy to come up with as long you don’t have many records and fewer properties on each of those objects. For example, in the code below if you want to test IPersonRepository’s GetAll method you will try to write dummy data as a List<Person>(). I am using FakeItEasy as my mocking framework.

var repo = A.Fake();

var people = new List(){ 
	new Person {FirstName = "Mitul", LastName = "Suthar"},
	new Person {FirstName = "Scott", LastName = "Smith"}
};
A.CallTo(() => repo.GetAll()).Returns(people);

1. What if you have more properties than just firstName and LastName? Typically in a real world you will have lots of properties and lot of different values which you will have tough time creating and remembering for your particular unit test.

2. What if you have entities that are related? While IPersonRepository’s GetAll() method is very basic but what if you want to test FindAllPeopleWhoDidNotPayBills() in which you are doing a LINQ query against different related tables and want to test if that LINQ query is returns appropriate data.  That LINQ query might look something like this.

var peopleWhoDidNotPayBills = from p in Persons
				join s in Sales on p.ID equals s.PersonId
				select new { p.FirstName, p.LastName, p.Email, s.OutStandingAmt};
							  

How are you going to generate fake data if you want to test this query.  A solution is you want to generate your own DbSet<Person> and DbSet<Sale> and you would make sure that the data would be related in those classes. Good luck coming up with that. 

Generally people would do data driven tests that will pull data from database, excel, csv file. At my work in our current application we have lots of existing tables (think more than 100) and lots of columns (more than 20) on each of those tables and it is just painful trying to hook up each table and property using our List initializer syntax. And on top of that our unit test would have become very slow.  In order to speed things up you want to have all your sample data hard coded in some helper method.

For each DbSet<T> we have a helper method that generates a DbSet<T> from hard coded List<T>.  That hard coded List<T> is generated automatically using LINQPad Extension method. In our team we love LINQPad and I wished if there was a way to generate List<T> based upon the LINQ query that we execute inside LINQPad. So here is the extension method.

Copy and paste inside your MyExtensions static class in My Queries section of LINQPad. If you write any extension method here it is available throughout LINQPad.

public static object GetClist(this object value, string className)
	  {
		var enumerator = (IList)value;
		Type typeList = value.GetType().GetGenericArguments()[0];  
		
		var listBuilder = new StringBuilder();
		listBuilder.AppendLine(String.Format("new List<{0}>() {{",className));				
		foreach (dynamic element in enumerator)
		{
		
			Type type = element.GetType();	
		
			var classBuilder = new StringBuilder();
			classBuilder.AppendLine(String.Format(@"new {0}() {{", className));
			List lstElements = new List();
			foreach (PropertyInfo p in type.GetProperties())
			{
				if (p.GetValue(element) != null)
				{				
					if (p.GetValue(element).GetType().Name =="DateTime")
					{
						lstElements.Add(String.Format(@"{0} = Convert.ToDateTime(""{1}"")", p.Name, p.GetValue(element)));
					}else if(p.GetValue(element).GetType().Name == "String"){
						lstElements.Add(String.Format(@"{0} = ""{1}""", p.Name, p.GetValue(element))); 
					}else if(p.GetValue(element).GetType().Name == "Decimal"){
						lstElements.Add(String.Format(@"{0} = {1}",p.Name, p.GetValue(element))); 
					}else if(p.GetValue(element).GetType().Name == "Int32"){
						lstElements.Add(String.Format(@"{0} = {1}",p.Name, p.GetValue(element))); 
					}else if(p.GetValue(element).GetType().Name == "Boolean"){
						lstElements.Add(String.Format(@"{0} = {1}",p.Name, p.GetValue(element))); 
					}else if(p.GetValue(element).GetType().Name == "Guid"){
						lstElements.Add(String.Format(@"{0} = new Guid(""{1}"")",p.Name, p.GetValue(element))); 
					}else{
						//Console.WriteLine("{0} = {1}, baseType: {2}", p.Name, p.GetValue(element), p.GetValue(element).GetType().Name);	
					}				
				}			
			}			
			for(int i = 0; i < lstElements.Count;i++){
				if (i == lstElements.Count - 1)
				{
					classBuilder.Append(String.Format("{0}",(lstElements.ToList()[i].ToString())));
				}else{
					classBuilder.AppendLine(String.Format("{0},",(lstElements.ToList()[i].ToString())));
				}
			}		
			listBuilder.AppendLine(classBuilder.ToString());
			listBuilder.AppendLine(@"},");		
		}
		var preFinal = listBuilder.ToString().Remove(listBuilder.ToString().LastIndexOf(','), 1);				
		var finalValue = String.Format(@"{0}}};", preFinal);
		Console.WriteLine(finalValue);
		return value;
	}							  

On how to use it.

1. Save your query. On saving it gets compiled and it gives you warning if you didn’t something wrong.

2. In LINQPad any anonymous query will not just work. You will have to do a ToList() and then GetCList(“Nameofclass”) method to get List<T> as C# text which you can copy and paste. 

3. Don’t forget to specify the type of the class you want to generate in the GetCList(“Person”);

4. Now you can copy that the generated list and then use it in your dummy data generator class.

5. If you want to add support for different data types then first you might want to know which property is not showing up.  Just uncomment this line in the extension and it will display which type is not listed.

   //Console.WriteLine("{0} = {1}, baseType: {2}", p.Name, p.GetValue(element), p.GetValue(element).GetType().Name);   

Then just add another if else and generate your own type initializer.

Since this is a LINQPad extension you can do a lot of mix and match and generate a lot sample data as C# text pretty quickly.  I am using AdventureWorks and using GetCList method against Persons. This is how it looks like.

var people = from p in Persons
select new
{
	p.BusinessEntityID,
	p.PersonType,
	p.NameStyle,
	p.Title,
	p.FirstName,
	p.MiddleName,
	p.LastName,
	p.Suffix,
	p.EmailPromotion,
	p.AdditionalContactInfo,
	p.Demographics,
	p.Rowguid,
	p.ModifiedDate
};
people.Take(2).ToList().GetClist("Person");

And the generated C# List is pretty basic nothing fancy here. But this is generated using the extension method.

new List() {
new Person() {
BusinessEntityID = 1,
PersonType = "EM",
NameStyle = False,
FirstName = "Ken",
MiddleName = "J",
LastName = "Sánchez",
EmailPromotion = 0,
Rowguid = new Guid("92c4279f-1207-48a3-8448-4636514eb7e2"),
ModifiedDate = Convert.ToDateTime("2/8/2003 12:00:00 AM")
},
new Person() {
BusinessEntityID = 2,
PersonType = "EM",
NameStyle = False,
FirstName = "Terri",
MiddleName = "Lee",
LastName = "Duffy",
EmailPromotion = 1,
Rowguid = new Guid("d8763459-8aa8-47cc-aff7-c9079af79033"),
ModifiedDate = Convert.ToDateTime("2/24/2002 12:00:00 AM")
}
};

Another pain point in this is if you are generating DataTime then getting the right DateTime for that particular event is tough. You have to think through different events that might have happened with that entity. For example, Lastvisited, ApprovedOn, InactivatedOn, PaymentPostedDate, etc.  These dates have to be related in real world business problems. By generating data this way you are sure it will be related and you don’t have to think about it. 

In LINQ using the let keyword you can generate different sets of data easily. You can generate related DbSet<T> in LINQPad easily but coming up related data manually is very time consuming and boring. If you made this far in a very silly post then thanks for that. Please let me how you are testing against DbSet<T> with lots of properties and if this could be improved upon.

Introducing TfsManager Some PowerShell cmdlets to work with Tfs

TfsManager is a project hosted on github by me.  The idea came about this project when I was fiddling around with Team Foundation Server Object Model.  Using the .dlls that are provided as part of Visual Studio installation or Team Explorer, you can pretty much do anything with TFS using code.  I wanted to explore what I can do with that Object Model.  At first sight these PowerShell cmdlets don’t seem to solve a significant problem that can’t be solved with traditional GUI or TF.exe.  But let’s hold on to that thought for a while.  Let’s get started.

github url: https://github.com/mitulsuthar/TfsManager

You can fork it, submit issues and pull requests.  I will be happy to work with you. 

Install the PowerShell module by using this single command. 

(new-object Net.WebClient).DownloadString("https://raw.github.com/mitulsuthar/TfsManager/master/tools/InstallTfsManager.ps1") | iex

This will copy .dll file into your Modules folder under a directory called TfsManager.Cmdlets.

Note: There is no help file included right now so traditional help cmdlets won’t return any examples.

1. Get all the cmdlets available for this module.

PS C:\> get-command -module TfsManager.cmdlets

 

2. Get all the Tfs Server Url configured on your machine.

PS C:\>Get-TfsServer

Note: I apologize for some whitening on the output as I had to hide some sensitive information.

If you have just one Tfs Server then it will return a String object and if you have more than one then it will return array of string.

3. Get all the Team Project Collection for a particular Tfs Server. 

PS C:\> Get-TfsProjectCollection -ServerUri (get-TfsServer)[0]

From (2) you can see I am referencing the first Tfs Server which is the VisualStudio hosted version of Tfs.  If your credentials aren’t cached then it will ask you to first connect to it using a Microsoft Account.  After you provide your credentials you will see your Tfs Project Collections.

Alright. Here comes why these cmdlets are of interest. Because they provide you the full .Net object work inside PowerShell.  Let’s get the returned Project Collection into a variable $col and do a get-member.

Ah ! See all the methods and properties you can get to.  But I know you are not happy with this.  Let’s do something more.  So if you have worked with Tf.exe then you might know that Tf.exe commands work if you provide TeamProjectcollectionUri or if you are currently on a path that is mapped to your project folder.  I am going to navigate to a project folder locally inside PowerShell and then try to run some cmdlets.

3. Get mapped workspace for a particular path

PS C:\Your Mapped Folder Path>Get-TfsWorkspace

Aha! It returns a .Net object of Type Workspace.  Let’s do Get-Member on the workspace and see if you are impressed or not.  There are more methods and properties that I can’t show in the screenshot below.  You can do all the interesting stuff with just Get-TfsWorkspace command. You can get reference to objects returned by these cmdlets and write your own script to do some work. 

Actually this project TfsManager is a by product of something I am working with managing Tfs Builds. I will share that project some time later.  But talking of Builds let find out if there is something for us in these cmdlets.

4. Get all the BuildDefinitions associated with the currently mapped folder or a project.

PS C:\Your Mapped Folder Path>Get-TfsBuildDefinition

Warning: This will enumerate all the properties of a BuildDefinition  and it will clutter your output. 

Again Get-Member to the rescue.  After getting a reference to our build definition we perform the following methods on that BuildDefinition.  For shorthand do Get-TfsBuildDefinition | Select Name

Alright one last command before I conclude this post.

5. Get all the builds for a particular mapped path

PS C:\Your Mapped Folder Path>Get-TfsBuild

This command will first get the Team Project and find all the builds. Again be careful it will enumerate all the properties for the object IBuildDetail.  So a Get-Member again to see all the methods.

To see just the important information you can do the following command.

Sometimes if you have lots of builds older than certain date or meet certain criteria and want to just delete them then you can do that easily in a script.

6.  Ok I couldn’t resist this one.  Not very useful but you can still play with it. For a particular path you can get PendingChanges too.

Get-TfsPendingChanges | select localitem

So that’s about it for now.  I have lots of hypothetical scenarios coming in my mind that I can do with these cmdlets but I want to hear from you on what do you think about this project and what are the bugs and issues. 

Work with TF.exe from Visual Studio Package Manager Console

I work with Team Foundation Server and I have found sometimes it takes lots of mousing [I mean mouse movements] to accomplish a certain task.  While if you don’t know then all the commands that are available to us from Visual Studio are also available from the command line.  Oh! wait it is the Visual Studio command prompt. So there is just one thing you will have to do in order to make TF.exe available from almost everywhere.  Add the following path to your system environment variable and TF.exe will be now accessible from the command prompt. 

C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE

[This is the path for Visual Studio 2013 installed on my machine. If you are using VS2012 then change 12.0 to 11.0 version]

After this you can just do like this Win + R + cmd and Enter.  Then just type TF /?

Since this is available from normal command prompt what would prevent us from opening PowerShell and try there. 

Oh! It works there.  Btw this was my favorite approach to work with Team Foundation Server until I thought what if this would work from Visual Studio Package Manager Console which is using PowerShell. 

Awesome. So why I like this approach? If I am working on something in regular PowerShell and to find pending changes I have to navigate to my working directory to do tf status.  But with Package Manager Console I can just do tf status and it gives me pending changes for just that project and that particular branch that I am in.  Try doing pwd it will give the current location of your project.  And since docking in VS is supported for all the windows I just dock Package Manager Console to another monitor.

I use regular PowerShell and Package Manager Console for committing changes to TFS interchangeably. 

PSBuild makes working with MSBuild.exe easier from PowerShell

PSBuild is a project started by Sayed Hashimi to simplify working with MSBuild.exe from PowerShell. It is hosted on gitHub.  I really like this project because I love PowerShell and if I can do things in PowerShell then I am more inclined to use PowerShell first. 

PSBuild installs into your Modules folder and is available once you install it by the following very simple command. I love projects that go one extra mile to make setup as easier as possible.  Open PowerShell and copy the following command and paste into console by just right clicking your mouse.

(new-object Net.WebClient).DownloadString("https://raw.github.com/ligershark/psbuild/master/src/GetPSBuild.ps1") | iex

Once you have installed it, you can list all the commands available by this command.

Get-Command –Module PSBuild

The most important command we are interested here is Invoke-MSBuild cmdlet.

Get-Help Invoke-MSBuild

There are lots of examples on how to use Invoke-MSBuild

Get-Help Invoke-MSBuild -Examples

MSbuild is very powerful and rightly so as it is used to build visual studio projects.  With Visual Studio 2013 MSbuild is being shipped with Visual Studio and is now referred to as MSBuild 2013. Previously it was shipped with .Net framework and versioning was also different now the version is bumped up to 12.0 from 4.0 to match with VS2013 (it is 12.0). It also comes as separate package known as Build Tools Package. This allows you to install the Build Tools Package as standalone and create a light weight build server. Path to MSBuild has also been changed and Invoke-MSBuild makes it easier for us to not know these different versions and hunt around for paths.  If you want to get MSBuild.exe then you can use one of the cmdlets as

Get-MSbuild

Another one to check the versions

& (Get-MSBuild) /version

If you want to get the MSBuild.exe help then

& (Get-MSBuild) /help

As you have noticed,  you can invoke MSbuild like this if you want but Invoke-PowerShell is better because you don’t have to second guess the syntax of properties and target that you will specify.  You pass properties as hash table. Let’s do something interesting here.

I am going to create an empty asp.net mvc website and try to build it and then publish to IIS website using a publish profile.

Simple call to build the project.
PS C:\BadSourceCodes\demoWebApp\demoWebApp> Invoke-MSBuild .\demoWebApp.csproj

You can get the latest log file

PS C:\Get-PSbuildLog

And if you want to open the log file then

PS C:\Open-PSBuildLog

Since this is a web project you can setup web publishing inside visual studio.  You can see these series of article on how to do web publishing here. I have created a website in IIS7 and setup Web Deploy publishing for that website.  Thanks to Sayed for his blog post - I was able to publish my website to the IIS website I by passing these parameters.

>Invoke-MSbuild .\demoWebApp.csproj –DefaultProperties (@{'Configuration'='Release';'DeployOnBuild'='true';'PublishProfile'='Default Settings';'username'='myusername';'password'=’password';'AllowUntrustedCertificate'='true'})

With PSBuild you can create a script that you can schedule to auto run on your build server that will automatically build the project and deploy the web project to IIS website.

Go check out the project on github and happy automating.

Making TypeScript compile on TFS2010 Server

Issue:

So the thing is TypeScript is not included with VS2013 out of the box.  I thought it was included but it not I had to download “TypeScript for VS2012 and VS2013” and install locally. When you try to build your web project locally with VS2012/13 then it will produce .js file.  So that’s well and good.  But when you try to build on build server it will not work. As expected because it doesn’t have the Targets to compile .ts file. 

Resolution:

1.  Like most build environments if you have VS2012/13 already installed on your build server then download the “TypeScript for Visual Studio 2012 and 2013” and install on the build server.  This will build the .ts file into .js file on the build server.  However, keep in mind that when you build your solution locally this file (.js) is also created locally and this file should not be included in your source control folder just like normal practice we don’t include .dll files from the bin folder or .exe files from obj folder.  Exclude that file or convert that file into .ts file and delete the duplicate file.

2. Another way to compile TypeScript files on the Build Server is detailed by Tom DuPont – And that is to create separate folder on Source Control and reference the targets from there.

I am sure there are other ways but I found method 1 to be easiest as I already have VS2012 and VS2013 installed on the build server.