Making to make a web crawler and a link tree purely in Javascript

In one of my assignments I have to make a web crawler, purely in JavaScript. That means, the input will be an URL and output will be a tree of all the links starting from that page. I used the plugin https://github.com/padolsey/jQuery-Plugins/tree/master/cross-domain-ajax/ which uses YQL and it worked for fetching all the links on the site. (Like I did for the Google's Homepage , http://deepakpathak.in/langoor/linkfinder.html). However I am having trouble making a tree structure out of it. Is there any other simpler way to make a tree of links of a given website, and that in Javascript?

Answers


You're not mentioning what kind of tree you want to do, should the output be a tree list component on a website of some sort or do you want to put it in a database?

However you can use plain old javascript objects and arrays to create your tree (look up any notes on the tree data structure to see how they work there are many ways to represent them). To get you started a basic tree can look like this (with a createNode function):

var createNode = function(content) {
    return {
        'nodes': [], // children are put here
        'content': content // puts the content here
    };
};

var treeroot = createNode();
    // create the root node of the tree
    // content is undefined

treeroot.nodes.push(createNode(myTreeNode)); 
    // accesses the nodes array and pushes a new node into the root node
    // content in that node will be whatever "myTreeNode" is

You do have to write traversal algorithms yourself as javascript doesn't have any functions to handle trees. Alternatively create your tree using the DOM itself (as it is a tree data structure).


Need Your Help

Cassandra throwing org.apache.cassandra.config.ConfigurationException: Cannot locate dse.yaml

cassandra datastax-enterprise

I am trying to run sstable2json utility after starting cassandra server but it throwing following error.

Passing data from php session vertiable to ajax (with some jQuery in between)

php jquery ajax security

I want to pass the user id and username stored in the session variable in php with an ajax request.

About UNIX Resources Network

Original, collect and organize Developers related documents, information and materials, contains jQuery, Html, CSS, MySQL, .NET, ASP.NET, SQL, objective-c, iPhone, Ruby on Rails, C, SQL Server, Ruby, Arrays, Regex, ASP.NET MVC, WPF, XML, Ajax, DataBase, and so on.