HTML form has one javascript generated radio button. Is there a way around this using mechanize?

This form, for whatever reason, uses javascript's document.write to draw the radio buttons on the page, and I can't figure out how to circumvent it.

After some sleuthing around here and on Google, it seems that you can manually recreate the form field with Mechanize, and then submit it as usual.

So, I've set up the code as follows:

br = mechanize.Browser(factory=mechanize.RobustFactory())
response = br.open(url)
br.select_form(nr=0)
br.form.set_all_readonly(False)
br.form.new_control('radio', 'DATASOURCE', {'value':'FILE', 'checked':'true'})
br.form.add_file(open('weather_info.csv'), 'text/csv', 'weather_info.csv', name='FILENAME')
br.form.fixup()
response = br.submit() 

Now, if I print the form, it does appear as one of the fields at the bottom.

<HiddenControl(CGIREF=/calludt.cgi/DDFILE1)>
  <HiddenControl(USE=MODEL)>
  <HiddenControl(MODEL=CM)>
  <HiddenControl(CROP=apples)>
  <HiddenControl(METHOD=SS)>
  <HiddenControl(UNITS=E)>
  <HiddenControl(LOWTHRESHOLD=50)>
  <HiddenControl(UPTHRESHOLD=88)>
  <HiddenControl(CUTOFF=H)>
  <SelectControl(COUNTY=[])>
  <CheckboxControl(ACTIVE=[*Y])>
  <SelectControl(FROMMONTH=[1, 2, *3, 4, 5, 6, 7, 8, 9, 10, 11, 12])>
  <SelectControl(FROMDAY=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, *15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31])>
  <SelectControl(FROMYEAR=[2014, *2013, 2012, 2011, 2010, 2009, 2008, 2007, 2006, 2005, 2004, 2003, 2002, 2001, 2000, 1999, 1998, 1997, 1996, 1995, 1994, 1993, 1992, 1991, 1990, 1989, 1988, 1987, 1986, 1985, 1984, 1983, 1982, 1981, 1980, 1979, 1978, 1977, 1976, 1975, 1974, 1973, 1972, 1971, 1970, 1969, 1968, 1967, 1966, 1965, 1964, 1963, 1962, 1961, 1960, 1959, 1958, 1957, 1956, 1955, 1954, 1953, 1952, 1951])>
  <SelectControl(THRUMONTH=[1, 2, 3, 4, *5, 6, 7, 8, 9, 10, 11, 12])>
  <SelectControl(THRUDAY=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, *12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31])>
  <SelectControl(THRUYEAR=[2014, *2013, 2012, 2011, 2010, 2009, 2008, 2007, 2006, 2005, 2004, 2003, 2002, 2001, 2000, 1999, 1998, 1997, 1996, 1995, 1994, 1993, 1992, 1991, 1990, 1989, 1988, 1987, 1986, 1985, 1984, 1983, 1982, 1981, 1980, 1979, 1978, 1977, 1976, 1975, 1974, 1973, 1972, 1971, 1970, 1969, 1968, 1967, 1966, 1965, 1964, 1963, 1962, 1961, 1960, 1959, 1958, 1957, 1956, 1955, 1954, 1953, 1952, 1951])>
  <FileControl(FILENAME=weather_scrape.csv)>
  <IgnoreControl(Submit=<None>)>
  <RadioControl(DATASOURCE=[*FILE])>>

However, upon running the code and submitting the form nothing happens; it stays on the same page it started on.

Are there any ways to get around this using Python?

Edit:

I realized that the submit button is also javascript; it has an onclick event tied to it so.. I may be out of luck as far as a python solution goes..

<input type="button" name="Submit" value="Continue" onclick="SetDDinfo()">

I tried adding a button manually to the form as is shown here.

br.form.new_control('submit', 'Button', {}) 

But still no luck.

Submit's onclick method calls the following function:

function SetDDinfo(){
     var DDstuff = new Array(21);
     var checksok = true;
     // alert("SetDDinfo DDparms: " + DDparms);
     for (var i = 0; i < 21; i++){
       DDstuff[i] = DDparms[i];
     }
     // This section for MODEL
        document.DDCOMPUTE.USE.value = "MODEL";
        DDstuff[4] = "MODEL"
        with(document.DDCOMPUTE) {
          DDstuff[5] = "E";  // english units
          DDstuff[6] = "";  // lower threshold (null)
          DDstuff[7] = "";   // upper threshold (null)
          DDstuff[8] = "SS"; // dd method: (default)
          DDstuff[9] = "H"; // cutoff: (default)
          DDstuff[10] = "CM";         // organism model code: CM, NOW, etc.
          if (COUNTY.selectedIndex < 0) {
             DDstuff[11] = "";
          } else {
             DDstuff[11] = COUNTY.options[COUNTY.selectedIndex].value;  // county
          }
          if (DATASOURCE[0].checked == "1") {
             DDstuff[13] = "STATION";  // data from database
             DDstuff[14] = "";
          } else {
             if (DATASOURCE[1].checked == "1") {
                DDstuff[13] = "FILE";  // data from user file
                DDstuff[14] = FILENAME.value;
             } else {
                DDstuff[13] = "ENTRY";  // data from user entry
                DDstuff[14] = "";
             }
          }
        }
        // MODEL error checks. Return false to abort SUBMIT.
        if (DDstuff[11] == "" && DDstuff[13] == "STATION") {
           alert ("ERROR: You must select a County.");
           checksok = false;
           return checksok
        }
        if (DDstuff[10] == "") {
           alert ("ERROR: You must select a Model.");
           checksok = false;
           return checksok
        }
        if (DDstuff[13] == "FILE" && DDstuff[14] == "") {
           alert ("ERROR: You must specify a filename.");
           checksok = false;
           return checksok
        }
        // Wx source error checks. Return false to abort SUBMIT.
        if (DDstuff[11] == "" && DDstuff[13] == "STATION") {
           alert ("ERROR: You must select a County.");
           checksok = false;
           return checksok
        }
        if (DDstuff[13] == "FILE" && DDstuff[14] == "") {
           alert ("ERROR: You must specify a filename.");
           checksok = false;
           return checksok
        }

     DDstuff[14] = "";     // Don't bother saving filename, since not displayed and must be re-specified.
     var DDitems = DDstuff.join(",");
     SetCookie ('DDinfo', DDitems, exp);
        // alert ("DDstuff[13]: "+DDstuff[13]);
        if (DDstuff[13] == "STATION") {
           document.DDCOMPUTE.action = "/calludt.cgi/DDSTATIONLIST";
           document.DDCOMPUTE.method = "GET";
           document.DDCOMPUTE.encoding = "application/x-www-form-urlencoded";
        } else {
           if (DDstuff[13] == "FILE") {
              document.DDCOMPUTE.action = "/WEATHER/textupload.cgi";
              document.DDCOMPUTE.method = "POST";
              document.DDCOMPUTE.encoding = "multipart/form-data";
           } else {
              document.DDCOMPUTE.action = "/calludt.cgi/DDENTRY1";
              document.DDCOMPUTE.method = "GET";
              document.DDCOMPUTE.encoding = "application/x-www-form-urlencoded";           }
        }
        // alert ("document.DDCOMPUTE.action = " + document.DDCOMPUTE.action);
        document.DDCOMPUTE.submit()
     return checksok
   }  // ................................................

I'm not fluent in javascript, but the first half of the function seems to be dealing with a section of the form that I'm not interested in -- when the radio button is in position one. The second bit is tough for me to follow, but it's making a POST to /WEATHER/textupload.cgi I'll have to check the out the traffic tomorrow. Maybe I can recreate what it needs via urllib2 or something?

Answers


Need Your Help

Linker to link Functions found in different Code space but same Flash

c linker

I am using HCS08 micro-controller and Code Warrior as Development Environment.

How to implement SPDY with Rails 3.2.2 on Heroku?

ruby-on-rails spdy

I'm hearing that SPDY is where things are likely headed, and I'd like to try to use it with a Rails site I'm running. I haven't been able to find any tutorials however, and the one gem I found does...

About UNIX Resources Network

Original, collect and organize Developers related documents, information and materials, contains jQuery, Html, CSS, MySQL, .NET, ASP.NET, SQL, objective-c, iPhone, Ruby on Rails, C, SQL Server, Ruby, Arrays, Regex, ASP.NET MVC, WPF, XML, Ajax, DataBase, and so on.