blah blah blah is here! blah blah » Close

1
Artificial_Object
753
574 views

Look for a way to optimize with RegExp, HttpWebRequest using C# 4.0

up1down
link

Hi, you could say I've just started playing around.
My idea here is I want to be able to use the "define:" function in Google searchers from any of my applications.
example : type "define:Test" into google and you would know what I am talking about.

I have written something that does the job...however may be someone with more knowledge of C# can suggest to me ways to optimize or improve this ?

okay.
First I create.

first : iPlugin.cs contains public interface iPlugin
this is just a base for all my plugins...nothing special

Second : GoogleDefine.cs is what does the actual work.
notice I created 2 Regular expressions ( simple ones ) for the start and end tag of the unordered list.

The Process :
> Create a web request to google with the search parameter
> Get the response
> Get the Stream
> Read the stream into an Object type variable called Output
> Now use a regular expressions to cut off all html other than the <ul>...</ul> that I want by means of RegExp.Split()
I store the new value back into the Output Object variable.

what I don't like is that I am using 2 regular expressions and I am cutting a string twice and filling in some cut of characters...I'm sure there should be a way around this.


==========================================================================================

/* iPlugin.cs */
using System;


    public interface iPlugin
    {
        object Input { get; set; }
        object Output { get; set; }
        void Process();
    }

==========================================================================================
/* GoogleDefine.cs */

using System;
using System.Text;
using System.Net;
using System.Text.RegularExpressions;
using System.IO;

   public class GoogleDefine : iPlugin
    {
        public object Input { get; set; }
        public object Output { get; set; }
        public void Process()
        {
            Regex ulstart = new Regex("<ul ");
            Regex ulend = new Regex("</ul>");
            HttpWebRequest Request = (HttpWebRequest)WebRequest.Create("http://www.google.co.za/search?hl=en&q=define:"+Input.ToString()+"&btnG=Search");
            HttpWebResponse Response = (HttpWebResponse)Request.GetResponse();
            Stream s = Response.GetResponseStream();
            StreamReader r = new StreamReader(s, Encoding.ASCII);
            Output = r.ReadToEnd();
            object[] obj = ulstart.Split(Output.ToString());
            Output = "<ul "+ obj[1];
            obj = ulend.Split(Output.ToString());
            Output = obj[0] + "</ul>";
            r.Close();
            s.Close();
            Response.Close();
            Request.Abort();
        }
    }

==========================================================================================
Just for the purpose of those who want to use this code I call the code like this :

// get the class
GoogleDefine GD = new GoogleDefine();
// set the input
GD.Input = "Hello";
// process information
GD.Process();
// get the output ( you don't need to use an application variable )
Application["GoogleResponse"] = GD.Output;

c# regexp httpwebreqeuest optimize google
  • asked one year ago
  • edited one year ago
add comment
last answered one year ago

2 answers

Sort:
up1down
tick
link

Going back to your original attempt to do this using Regex, you can in fact extract the opening and closing <ul> tags and everything in between, using a single regular expression:

public class GoogleDefine : iPlugin
{
   public object Input { get; set; }
   public object Output { get; set; }

   public void Process()
   {           
      HttpWebRequest Request = (HttpWebRequest)WebRequest.Create("http://www.google.co.za/search?hl=en&q=define:"+Input.ToString()+"&btnG=Search");
      HttpWebResponse Response = (HttpWebResponse)Request.GetResponse();
      Stream s = Response.GetResponseStream();
      StreamReader r = new StreamReader(s, Encoding.ASCII);
      
      Output = Regex.Match(r.ReadToEnd(), "<ul .*?</ul>").Value;      

      r.Close();
      s.Close();
      Response.Close();
      Request.Abort();
   } 
}

To explain how that works:

. matches any character
* specifies that there are 0 or more such characters
? makes the matching 'lazy' i.e. it stops just before the following </ul> rather than chomping right through it.

  • answered one year ago by:

    vulpes
    17279
  • edited one year ago
(x)
Artificial_Object
753

Hi thank you. it seems a good regex is far more efficient :) as I said I am kind just starting with this...know any web guides on understanding Regex ?

(x)
Artificial_Object
753

also : thanks for the quick Regex explanation you added.

(x)
vulpes
17279

Even though it's not specific to .NET, I've always thought that the best online guide for understanding regular expressions is at http://www.regular-expressions.info/. The C# 3.0 (or 4.0) in a Nutshell books also have a very good chapter on them. Incidentally, the '?' is only really necessary if you have (or could have) more than one set of <ul> tags. If you know you only have one, it'll work fine without the '?' though it does no harm to leave it in.

add comment
up0down
link

I have improved the code in the GoogleDefine class. I now use string index positions instead of RegEx.
if you feel this is less efficient please let me know...if you have a better solution. let me know...if you have suggestions and improvements...let me know.

public class GoogleDefine : iPlugin
    {
        public object Input { get; set; }
        public object Output { get; set; }
        public void Process()
        {
            /*
            Regex ulstart = new Regex("<ul ");
            Regex ulend = new Regex("</ul>");
             * */
            HttpWebRequest Request = (HttpWebRequest)WebRequest.Create("http://www.google.co.za/search?hl=en&q=define:"+Input.ToString()+"&btnG=Search");
            HttpWebResponse Response = (HttpWebResponse)Request.GetResponse();
            Stream s = Response.GetResponseStream();
            StreamReader r = new StreamReader(s, Encoding.ASCII);
            Output = r.ReadToEnd();
            int old_len = Output.ToString().Length;
            int start = Output.ToString().IndexOf("<ul ");
            Output = Output.ToString().Substring(start);
            int end = Output.ToString().IndexOf("</ul>");
            Output = Output.ToString().Substring(0, end);
            //throw new Exception(old_len.ToString() + " : " + Output.ToString().Length.ToString() + " : " + end + " : " + ol);
            /*
            object[] obj = ulstart.Split(Output.ToString());
            Output = "<ul "+ obj[1];
            obj = ulend.Split(Output.ToString());
            Output = obj[0] + "</ul>";
             * */
            r.Close();
            s.Close();
            Response.Close();
            Request.Abort();
        }
    }

(x)
Artificial_Object
753

Yes...I was over-complicating a simple solution.

(x)
Artificial_Object
753

oops this line was removed : [code] int old_len = Output.ToString().Length; [/code] was during some of my crazy tests...

add comment
Feedback