.Net Stream Programming

.Net view programmable data byte[] as follows:
 (1) Backing store, disk, memory, network socket
 (2) Base Stream: FileStream, MemoryStream, NetworkStream, IO.PipeStream
 (3) Decorator Stream: GZipStream, CryptoStream, BufferedStream, DeflateStream
 (4) StreamAdaptor: Stream, Binary, Xml Reader/Write, 
(5) Network Stream does not support Buffering, so a decorator BufferStream is used to wrap around it to support streaming -- typical Decorator Pattern.
 
As conformed to general networking concept, it provide pipes, chunking, Request/Response pattern
and raw network socket.

            var server = new NamedPipeServerStream("p1");
            var client = new NamedPipeClientStream("p1");

            server.WaitForConnection(); // wait a client to connect.
            server.WriteByte(123);
            client.Connect();
            client.ReadByte();

            var svr = new NamedPipeServerStream("p1", PipeDirection.InOut, 1, PipeTransmissionMode.Message);
            var clnt = new NamedPipeClientStream("p1");

            clnt.Connect();
            var msg = Encoding.UTF8.GetBytes("This is a message");
            clnt.Write(msg, 0, msg.Length);

            // server side chunk reading
            StringBuilder sb = new StringBuilder();
            var buf2 = new byte[10];
            svr.WaitForConnection();
            do
            {
                svr.Read(buf2, 0, buf2.Length);
                sb.Append(Encoding.UTF8.GetString(buf2));
                Array.Clear(buf2, 0, buf2.Length);

            } while (!svr.IsMessageComplete);

            // Web Stream
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://myserver/1.aspx");
            var requestStream = request.GetRequestStream();
            requestStream.Write((new ASCIIEncoding).GetBytes("param1=greeting&p2=9"));
            requestStream.Close();
            var response=request.GetResponse();
            StreamReader sr = new StreamReader(response.GetResponseStream());

            //Web Client cannot read bc have to close stream to send
            WebClient wClnt = new WebClient();
            wClnt.OpenWrite("http://..").Close(); // has to call close to send
            Stream s3 = wClnt.OpenRead("http://.."); // must re-open to get response


Tcp Raw Socket -- Cannot support UDP since it is not Connection Oriented

            var ip = IPAddress.Parse("127.0.0.1");
            TcpListener listener = new TcpListener(ip, 5200);
            listener.Start();

            Socket socket=listener.AcceptSocket(); // block
            NetworkStream s = new NetworkStream(socket);
            int byteRead = 0;
            byte[] buf = new byte[100];
            do
            {
                byteRead= s.Read(buf, 0, buf.Length); 
            } while (byteRead!=0);
            socket.Close();

            TcpClient client = new TcpClient("localhost", 5200);
            NetworkStream ns=client.GetStream();
            byte[] data = new byte[100];
            (new Random()).NextBytes(data);
            ns.Write(data,0,data.Length);
            ns.Close();
            BufferedStream bs = new BufferedStream(ns);
            bs.Write(data, 0, data.Length);

using HttpClient for async streaming

        async static void Get()
        {
            var client = new HttpClient();
            var resp = await client.GetAsync("http://www.cnn.com");
            resp.EnsureSuccessStatusCode();
            Console.Write(await resp.Content.ReadAsStringAsync());
        }

Raw byte[] chunk reading
        static byte[] ReadBytes(Stream s)
        {
            var s_threadSafe = Stream.Synchronized(s);

            byte[] buf = new byte[s_threadSafe.Length];
            int totalRead = 0; //already read.
            int chunk = 1;
            while (totalRead < s_threadSafe.Length && chunk>0)
            {
                chunk = s_threadSafe.Read(buf, totalRead,buf.Length-totalRead);
                totalRead += chunk;
            }

            return buf;
        }

a few more things about Stream adaptor
--- Wrap streams: new BinaryReader(s); StreamWriter(fs)
--- Need to sync up: StreamWriter.Flush(), StreamReader.DiscardBufferedData();
--- Data Layout match BinaryReader.ReadInt32 vs ReadInt16
--- For SSD, FileOptions.WriteThrough turnoff OS CachManager Caching, but not recommend otherwise.