SGML Reader - convert HTML to XHTML

This is a web - based implementation of converting HTML to well-formed XHTML using a solution which could be found in the past at the following location:
http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=B90FDDCE-E60D-43F8-A5C4-C3BD760564BC
There is one problem with this location, Microsoft closed gotdotnet.com so I decided to upload my adapted version. Maybe somebody knows how is the creator of this library so I can add this here.

You can download it from here: www.ronischuetz.com/download/sgmlreader.zip

Check if server is available before you make requests

I had to implement a fallback system in case one of my primary servers is not available anymore because of any reason. There were no option to access servers over a loadbalancer or something else then a direct routing to each server.

After some compassion's how much time I gone waste for each request between usage of sockets or HttpWebRequest / WebRequest I decided to work directly with Sockets.

The implementation I decided to go for were quite easy (remember: keep it simple as possible) and its fast like hell - 9ms I'm loosing each check if the server is available or not. We use 2 servers - so I have to remember that there is a total of 18ms I'm loosing each time we have a running cycle to analyse data. In general my tests gave showed me that 5 of the 9 ms are used for DNS.GetHostEntry(ServerName) - if you can use IP-Addresses then I suggest to do it that way. Check out one of my previous post about this DNS.GetHostEntry(ServerName) - IPAddress.Parse("127.0.0.1") vs. Dns.GetHostEntry("127.0.0.1")

With those ~20ms we can leave very good because a cycle needs right now something around 2.5min and it can be a max. of 15min. So how to solve this is the question, here the codesample from MSDN which I adapted and using to validate if my servers are online: http://msdn.microsoft.com/en-us/library/system.net.sockets.socket.aspx

Formatted Code is available here: http://www.ronischuetz.com/code/CheckIfServerIsAvailable.htm

   1:      private static bool ConnectSocket(string server, int port)

   2:      {

   3:          Stopwatch sp = new Stopwatch();

   4:          IPHostEntry hostEntry = null;

   5:          bool result = false;

   6:          sp.Start();

   7:          // Get host related information.

   8:          hostEntry = Dns.GetHostEntry(server);

   9:          sp.Stop();

  10:          Log.Info(string.Format("ConnectSocket->Dns.GetHostEntry({0}) needed: {1}ms ",server, sp.ElapsedMilliseconds));

  11:          sp.Reset();

  12:   

  13:          // Loop through the AddressList to obtain the supported AddressFamily. This is to avoid

  14:          // an exception that occurs when the host IP Address is not compatible with the address family

  15:          // (typical in the IPv6 case).

  16:          foreach (IPAddress address in hostEntry.AddressList)

  17:          {

  18:              sp.Start();

  19:              IPEndPoint ipe = new IPEndPoint(address, port);

  20:              Socket tempSocket = new Socket(ipe.AddressFamily, SocketType.Stream, ProtocolType.Tcp);

  21:              tempSocket.ReceiveTimeout = 1000;

  22:              tempSocket.SendTimeout = 1000;

  23:              // this is very important!!! Do not delete this.

  24:              tempSocket.NoDelay = true;

  25:              

  26:              try

  27:              {

  28:                  tempSocket.Connect(ipe);

  29:                  #region Status Check

  30:                  if (tempSocket.Connected)

  31:                  {

  32:                      #region Connnected Section

  33:                      result = true;

  34:                      if (tempSocket != null)

  35:                      {

  36:                          try

  37:                          {

  38:                              tempSocket.Shutdown(SocketShutdown.Both);

  39:                              tempSocket.Close();

  40:                          }

  41:                          catch (Exception ex)

  42:                          {

  43:                              // do nothing - only release resources and don't let the system fuck up

  44:                          }

  45:                          finally

  46:                          {

  47:                              tempSocket = null;

  48:                          }

  49:                      }

  50:                      #endregion Connnected Section

  51:                      break;

  52:                  }

  53:                  else

  54:                  {

  55:                      continue;

  56:                  }

  57:                  #endregion End Status Check

  58:              }

  59:              catch (Exception ex)

  60:              {

  61:                  string title = string.Format("Could not connect to server:{0}:{1} -  " + Environment.NewLine + ex.Message + Environment.NewLine + ex.StackTrace, server, port);

  62:                  Log.Info(title);

  63:                  Log.Fatal(title);

  64:                  result = false;

  65:              }

  66:   

  67:              sp.Stop();

  68:              Log.Info(string.Format("ConnectSocket->hostEntry.AddressList({0}) needed: {1}ms to connect to server ", ipe.Address.ToString(), sp.ElapsedMilliseconds));

  69:              sp.Reset();

  70:          }

  71:          return result;

  72:      }

Threaded Asynchronous Tcp Server with a blocking Client

I don't know now how many prototypes I have written in the past few weeks to verify which approach would fit best to scale and be most performance for indeXus.Net Shared Cache. To write a Client / Server Architecture which mostly will be N:M connectivity was not that easy as I thought in beginning.

Honestly I believe I have study every single C# sample I found on the common search engines and arrived to the point where I started to adapt the samples. Unfortunately 99% of founded samples are showing how to pass a string from the client to the server beside one blog which has written down a lot of theory and some very nice diagrams without to come up with code. I only can recommend you to read this blog up and down before you write even one single line of code: http://www.coversant.net/Coversant/Blogs/tabid/88/EntryID/10/Default.aspx. There are some very smart people around they have done great work.

After a while of research, I get to koders.com where I found a strip-down version of their product.

Another very useful page with a lot of sample code around .net is Mike Woodrings's .net Sample Page which contains a bunch of great examples for different domains.

I think I have tried any way to use Sockets now:

• Blocked Sockets
• Unblocked Sockets
• Poll Sockets
• Select

Every prototype I have done so far had his advantages and disadvantages. In the end of this post I will provide 2 downloads:

• prototype with poll sockets on client and server
• protptype with block sockets on client side and async server handling

Once I get comfortable with all the different Issues which come across the write the server part I decided to use Asad Aziz - Asynchronous Server Socket using C# Sample as my base for the server. Its very easy written but my knowledge on this issue was limited until I started to work on it quite intensive and I think I learned every day at least 10 new things :-)

Lets dive into different key parts of the client and the server. An additional issue I would like to mention here is that the whole code is 100% managed.

We gone start first with the server. As already mention the server is working Asynchronous which means we have to handle with IAsyncResult. Upon Server start we begin run the server within a different thread and the client cleanup will be handled by a TimerCallback to purge disconnected clients in case they not removed before upon disconnection.

ThreadStart serverListener = new ThreadStart(this.StartListening);
serverThread[0] = new Thread(serverListener);
serverThread[0].IsBackground = true;
serverThread[0].Start();
TimerCallback timerDelegate = new TimerCallback(this.CheckSockets);
this.lostTimer = new Timer(timerDelegate, null, SharedCacheTcpServer.timerTimeout, SharedCacheTcpServer.timeoutMinutes);

Since we have started now the serverListener we Bind the server IPEndPoint to the requested ip and port and start to listen for connections from clients. Once a connection is received it only will be destroyed in one of the following 2 cases:

1. The Socket has not been used for a certain amount of time
2. Client Socket get disconnected.

With this we avoid to much Server resources. As started we will use a ManualResetEvent to accept only one by one client. I have commented this part for testing purposes and the server started to throw memory exceptions. So keep it simple in the meaning, only start a new BeginAccept once you finished to move the client into AccecptCallback.

private void AccecptCallback(IAsyncResult ar)
{
// signal main thread to continue
this.allDone.Set();
Socket clientListener = ar.AsyncState as Socket;
if (clientListener != null)
{
Socket handler = clientListener.EndAccept(ar);
Console.WriteLine(@"Connected by client: {0}", handler.RemoteEndPoint);
StateObject state = new StateObject();
state.WorkSocket = handler;
state.AliveTimeStamp = DateTime.Now;

... and some more code ..

once you have done all setup for you StateObject (this is the object which contains all different data between the calls) we able to call BeginReceive. SharedCache Protocol between client and server keep simple: [messageLength][message] so if we are not able to get the whole message at once we have to call several times BeginReceive until we have received everything from the client and we able to run the custom stuff on the server.

Once we done with our reading we can start to process received package and process with it. To keep server resources (amount of threads) under control I decided to use Mike's Threadpool.


I think the 2 most important parts are within this message is to know that we need to read the header which indicates to message length and if we need to read more data for this message or do we can proceed it arrived data.

// check for header
if (state.ReadHeader) { .....


// check for message length -> TODO: how to check longer values as int count in list????
if (state.DataBuffer.ToArray().LongLength == state.MessageLength) { ...


as you can see there is even one open point since I use in my state object a List. I had some discussions with people on asp.net forums but I believe i can life right now with this
int.MaxValue as maximal message length ;-)


the next key position in above code print screen is that we use at this place again handler.BeginReceive() since we want to client connection keep opened as long as possible. Some previous tests had shown me while i open and close 1000 Sockets the time reduction to use always the same is more then 50%. Before we can post the request into ThreadPool we need to remove the Range of our message size [8 bytes] that's why we do: state.DataBuffer.RemoveRange(0,8); once we have posted it to the threadpool to pool handles the message sending the Echo to the waiting client.
HandleClientMessage is the place which is called after you get the free Thread from the ThreadPool - here you can manage your Server side stuff and once you done you can send it back to the Client. If you work within Winform environment a more event driven design would be correct but for sharedcache this would not be correct since the client is waiting for the server response.



in the above example we do nothing else then to set an Attribute of the object IndexusMessage to Successful.
The response from the server needs to be prepared, then also the client waits for the same struct [message length][message data]. Therefore we call UtilByte.CreateMessageHeader(with the msg.GetBytes() which returns a byte array) and we combine both values to 1 single byte array. since this is done we can free up resources and set the actual messageLength on the state so we know how much data we need to send to the client which is not less important.



Within the Send() method we start the async call: socket.BeginSend() or if we do not have anything to send back (this case does not happen here) we reset all state data and start to receive again without to destroy the state and the socket which is included in the state. The
same idea happens within the SendCallback() we call BeginSend() as long we have data to send and once we sent everything to the client we reset the state and start to receive again with socket.BeginReceive().

These are all important parts on the server side, I will wrap up the client in a different post within the next few days.

The 2 best code-solution I have found on the web are the following 2 links:

http://www.codeproject.com/KB/IP/Generic_TCP_IP_server.aspx
http://www.codeproject.com/KB/IP/AsyncSocketServerandClien.aspx

Both of them could not handle the case I needed so I hope people can use the Prototypes in this or another way. It would be great if you let me know if you are using them. So here are the downloads:

• Polling (Client and Server)
• Async Server and Block client with manager to connect to several servers

I really would like to say "thank you very much" to all contributors of the following articles and coding samples:

• Atif Aziz for all his good suggestions about how to handle the different issues
• Gil Y. - Generic TCP/IP Client / Server
• Andre Azevedo - An Asynchronous Socket Server and Client
• Mike Woodring - Custom Thread Pool
• Chris Mullins - Windows Sockets and Threading: How well does it scale?
• Mark Strawmyer - Communication over Sockets: Blocking vs Unblocking

.net C# code helper serie

I have no clue how many times you are writing the same code but I will start to use the blog for several helper classes I'm using all the time again and again and again.


This series starts with the System Information Helper class. This class helps to provide System Information for Windows Environments. The parts of the class are the following onces:

• Get .net framework information
• Get Operating System
• Get the name of the system machine
• Get the current executing location
• Get config files in executing location
• Get the current user

The code can be dowloaded from the following location: http://www.ronischuetz.com/code/SysInfo.cs.txt or just have a look at the formatted version: http://www.ronischuetz.com/code/sysinfo.cs.html.

If you send me some additional parts I will extend the post.