Skip to content
This repository has been archived by the owner on Apr 22, 2021. It is now read-only.

New documentation for chapter 1 #70

Open
wants to merge 33 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
e4dd38b
first commit of chapter 1: intro to web archiving, autopilot, and ins…
Aug 14, 2020
6177866
Added file for chapter 2 documentation
Aug 18, 2020
9d52ef2
install section of chapter 1 modified to include a way to install aut…
Aug 18, 2020
e986ba6
fixed in-line code to be code blocks and added some usage info
Aug 18, 2020
f0becd0
Added documentation for the usage of autopilot
Aug 19, 2020
c7b64e3
Added headlines to chapter 2, some usage information for Docker (may …
Aug 24, 2020
d29919e
last commit didn't work right, trying again
Aug 24, 2020
aa36246
Started the guide on how to write a behavior for people with little j…
Aug 26, 2020
34fb47e
Progress on the Format section for chapter 2 under Creating Your Firs…
Aug 26, 2020
a164403
Added more for format section, putting in code examples
Aug 26, 2020
9f978c9
fixing some stuff in format, adding some stuff in a new section expla…
Aug 27, 2020
1060548
wrote some stuff on the expected format of the export metadata object
Aug 28, 2020
200d604
added info on the match property in the metadata export
Aug 31, 2020
a9677ba
finished the tutorial on how to create a behavior in chapter 2
Sep 1, 2020
6ec4f75
added files for chapters 3, 4, and reference:API
Sep 2, 2020
959aa5a
added more steps into the "how to make a behavior" section of chapter…
Sep 9, 2020
41cc5e0
changed the title of the section after "setting up your file" to "get…
Sep 9, 2020
8259197
added stuff about using the `newBehavior` CLI in the getting started …
Sep 12, 2020
8a038f3
added directions for Testing your first behavior using the CLI
Sep 14, 2020
223c244
fixed some small formatting things
Sep 14, 2020
edbfea4
moved chapters to manual
Sep 14, 2020
d038ae7
wrote directions for how to set up config file in "testing your first…
Sep 15, 2020
2fa30a0
wrote the "fixing a broken behavior" section
Sep 15, 2020
d1eb53c
I don't remember what my changes were
Sep 16, 2020
ae14f2c
started the FAQ document
Sep 16, 2020
03c17d6
moved some advanced stuff from chapter 2 into chapter 3
Sep 16, 2020
005fa31
added a code of conduct to the manual
Sep 17, 2020
ce81db2
removed some stuff that was covered in chapter 2
Sep 21, 2020
4436c78
updated chapter 1
kyragaut Sep 23, 2020
422c4c4
moved locations of the chapters
kyragaut Sep 23, 2020
47ab4e8
moved locations of chapters
kyragaut Sep 23, 2020
681d0c6
Moved the chapters to the correct locations
kyragaut Nov 12, 2020
0ed0640
added bug report format file
kyragaut Nov 12, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions manual/FAQ.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# FAQ
## Is Autopilot open source?

Yes. All Webrecorder tools are licensed under open source licenses.

From the beginning, the goal of Webrecorder has been to build quality open source tools enable ‘web archiving for all’ to allow anyone with a browser to create their own web archives, and to accurately replay them at a later time.

## What does Autopilot cost?

Autopilot is free to use.

## What platforms does Autopilot run on?



## How can I contribute to Autopilot?

Read the chapter on [Hacking Autopilot](./chapter3.md) for a guide on contributing to Autopilot. We suggest that beginner developers take a look at [Chapter 2](./chapter2.md) for a simpler overview of how Autopilot works.

##Glossary of Terms


15 changes: 15 additions & 0 deletions manual/bug-report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#Bug Report Template
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could be in .github/ISSUE_TEMPLATE/bug-report.md with the following at the top of the file

---
name: Bug Report
about: Report an issue or problem
title: ""
labels: bug
assignees: ''

---

Fill out this template when reporting bugs.

##Description

##Stepts to reproduce
1.
2.
3.

##Current behavior (bug)

##Expected behavior (correct)

##Additional information
113 changes: 113 additions & 0 deletions manual/chapter1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Chapter 1


## What is web archiving?

Web archiving is the process of recording web resources. Various elements of the website such as HTML, scripts, images, videos, etc. can be recorded to preserve as much of the original resource as possible. By creating dynamic archives that provide a user with the same experience they would have if they accessed the original site, web archives can give much more information than a static screenshot.

Web archives may be utilized by future researchers, historians and the general public.

## Why Autopilot?

Autopilot is a tool that navigates a website similarly to how a human would, doing things like scrolling, clicking buttons, and playing videos. These actions executed through functions called "behaviors." Because Autopilot can go through the technically complicated aspects of a website, it is useful for recording high-fidelity websites. Furthermore, Autopilot is designed to be accessible for anyone to use, and anyone with little javascript knowledge to contribute to. This makes webarchiving available to everyone.

# Autopilot basics
Autopilot uses [behaviors](https://github.com/webrecorder/behaviors/blob/master/manual/behaviors.md) to collect metadata from websites. Behaviors are Javascript modules which perform a series of actions on a webpage in order to collect information.

## Installing Autopilot

To use this project you must first install its dependencies. You can do this via a package manager like [Yarn](https://classic.yarnpkg.com/en/docs/install/#mac-stable) or [npm](https://www.npmjs.com/).

```
$ yarn install
# or "npm install"
```

To install Autopilot, use Terminal to clone the Webrecorder repository:

**You must be using node 12~**

```
1. `git clone https://github.com/webrecorder/behaviors.git`
2. `cd behaviors`
3. `yarn build-watch`
4. `open new terminal window`
5. `bash scripts/init.sh`
```
If you wish to use this project via Docker, install [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/).

```
3. docker-compose build
4. docker-compose up -d
```

The Autopilot instance can be accessed in the browser at `http://localhost:8089`.

# Running Behaviors
1. In order to run the behaviors you will first need to build them:
```
npm run generate-runnable-behaviors
<!-- npm run build-dev -->
```

2. Then, go into the dist directory.
```
cd dist
```

3. In the dist directory you will see behavior files. Open the behavior file you want and copy the code.
```
open [filename]
```

4. Copy all of the behavior code and paste it into the developer console of a webpage. The console can be accessed in Chrome using the shortcut: Option + ⌘ + J (on macOS), or Shift + CTRL + J (on Windows/Linux).

You can also run behaviors using the CLI:

## CLI and API

The cli provides two commands `API` and `behaviors` and each command has its own options.

## Behaviors command

Execute `./bin/cli behaviors -b` to build the behaviors made available. This will build the behaviors using the behavior config file located in the root of this project.

The built behaviors, along with a behavior metadata file (`behaviorMetadata.js`), can be found in the `dist` directory which will be created for you if it does not exist in the root of this project.

## API command

To run the behavior API server execute `./bin/cli api --build-behaviors`.

This will start the API server after all behaviors provided by this project have been built.

If you have already built the behaviors using the `behaviors` command provided by the cli then you may omit the `--build-behaviors` flag.


Some configuration of the API server can be done via the environment variables listed below

* `BEHAVIOR_API_HOST`: the host the api server will use (e.g. 127.0.0.1)
* `BEHAVIOR_API_PORT`: the port the api server will listen on (e.g. 3030)
* `WR_BEHAVIOR_DIR`: path to the directory containing the built behaviors
* `WR_BEHAVIOR_METADATA_PATH`: path to the behavior metadata file
* `BUILD_BEHAVIORS`: should the api server build the behaviors before starting



# Docker
If you would like, you can use Docker to run Autopilot.

To build the wr-behaviors docker image (`webrecorder/behaviors:latest`) execute `docker-compose build`.

The image created is suitable for building behaviors and running the behavior api server.

The default configuration of the image is to run the api server, however you can substitute the default command with any of the cli commands listed previously.

For more information please consult the provided `Dockerfile` and `docker-compose.yml` files.

<!-- ## About Web Traffic
-->

<!-- ## Status page: pre-made behaviors
blocked by website
-->

Loading