Making to make a web crawler and a link tree purely in Javascript

In one of my assignments I have to make a web crawler, purely in JavaScript. That means, the input will be an URL and output will be a tree of all the links starting from that page. I used the plugin which uses YQL and it worked for fetching all the links on the site. (Like I did for the Google's Homepage , However I am having trouble making a tree structure out of it. Is there any other simpler way to make a tree of links of a given website, and that in Javascript?


You're not mentioning what kind of tree you want to do, should the output be a tree list component on a website of some sort or do you want to put it in a database?

However you can use plain old javascript objects and arrays to create your tree (look up any notes on the tree data structure to see how they work there are many ways to represent them). To get you started a basic tree can look like this (with a createNode function):

var createNode = function(content) {
    return {
        'nodes': [], // children are put here
        'content': content // puts the content here

var treeroot = createNode();
    // create the root node of the tree
    // content is undefined

    // accesses the nodes array and pushes a new node into the root node
    // content in that node will be whatever "myTreeNode" is

You do have to write traversal algorithms yourself as javascript doesn't have any functions to handle trees. Alternatively create your tree using the DOM itself (as it is a tree data structure).

Need Your Help

C circular dependency

c include c-preprocessor circular-dependency

I have this problem with circular dependency in C, I looked around the other questions about this topic but really couldn't find the answer.

arraylist remove eliminates following objects

java arraylist

I'm getting a very strange action in my code. I have an ArrayList of the following class.