20. Web Automation

Introduction

Fetching a URL from a Perl Script

Automating Form Submission

Extracting URLs

Converting ASCII to HTML

Converting HTML to ASCII

Extracting or Removing HTML Tags

Finding Stale Links

Finding Fresh Links

Creating HTML Templates

def templatefile(filename, fillings)
    aFile = File.new(filename, "r")
    text = aFile.read()
    aFile.close()
    pattern = Regexp.new('%%(.*?)%%')
    text.gsub!(pattern) {
        fillings[$1] || ""
    }
    text
end

fields = {
    'username' => whats_his_name,
    'count' => login_count,
    'total' => minutes_used
}
puts templatefile('simple.template', fields)

# @@INCOMPLETE@@
# An example using databases is missing

Mirroring Web Pages

Creating a Robot

Parsing a Web Server Log File

Processing Server Logs

Program: htmlsub

Program: hrefsub