This repository has been archived by the owner on Apr 22, 2021. It is now read-only.
-
-
Notifications
You must be signed in to change notification settings - Fork 6
New documentation for chapter 1 #70
Open
kyragaut
wants to merge
33
commits into
webrecorder:master
Choose a base branch
from
unknown repository
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
33 commits
Select commit
Hold shift + click to select a range
e4dd38b
first commit of chapter 1: intro to web archiving, autopilot, and ins…
6177866
Added file for chapter 2 documentation
9d52ef2
install section of chapter 1 modified to include a way to install aut…
e986ba6
fixed in-line code to be code blocks and added some usage info
f0becd0
Added documentation for the usage of autopilot
c7b64e3
Added headlines to chapter 2, some usage information for Docker (may …
d29919e
last commit didn't work right, trying again
aa36246
Started the guide on how to write a behavior for people with little j…
34fb47e
Progress on the Format section for chapter 2 under Creating Your Firs…
a164403
Added more for format section, putting in code examples
9f978c9
fixing some stuff in format, adding some stuff in a new section expla…
1060548
wrote some stuff on the expected format of the export metadata object
200d604
added info on the match property in the metadata export
a9677ba
finished the tutorial on how to create a behavior in chapter 2
6ec4f75
added files for chapters 3, 4, and reference:API
959aa5a
added more steps into the "how to make a behavior" section of chapter…
41cc5e0
changed the title of the section after "setting up your file" to "get…
8259197
added stuff about using the `newBehavior` CLI in the getting started …
8a038f3
added directions for Testing your first behavior using the CLI
223c244
fixed some small formatting things
edbfea4
moved chapters to manual
d038ae7
wrote directions for how to set up config file in "testing your first…
2fa30a0
wrote the "fixing a broken behavior" section
d1eb53c
I don't remember what my changes were
ae14f2c
started the FAQ document
03c17d6
moved some advanced stuff from chapter 2 into chapter 3
005fa31
added a code of conduct to the manual
ce81db2
removed some stuff that was covered in chapter 2
4436c78
updated chapter 1
kyragaut 422c4c4
moved locations of the chapters
kyragaut 47ab4e8
moved locations of chapters
kyragaut 681d0c6
Moved the chapters to the correct locations
kyragaut 0ed0640
added bug report format file
kyragaut File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
# FAQ | ||
## Is Autopilot open source? | ||
|
||
Yes. All Webrecorder tools are licensed under open source licenses. | ||
|
||
From the beginning, the goal of Webrecorder has been to build quality open source tools enable ‘web archiving for all’ to allow anyone with a browser to create their own web archives, and to accurately replay them at a later time. | ||
|
||
## What does Autopilot cost? | ||
|
||
Autopilot is free to use. | ||
|
||
## What platforms does Autopilot run on? | ||
|
||
|
||
|
||
## How can I contribute to Autopilot? | ||
|
||
Read the chapter on [Hacking Autopilot](./chapter3.md) for a guide on contributing to Autopilot. We suggest that beginner developers take a look at [Chapter 2](./chapter2.md) for a simpler overview of how Autopilot works. | ||
|
||
##Glossary of Terms | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
#Bug Report Template | ||
Fill out this template when reporting bugs. | ||
|
||
##Description | ||
|
||
##Stepts to reproduce | ||
1. | ||
2. | ||
3. | ||
|
||
##Current behavior (bug) | ||
|
||
##Expected behavior (correct) | ||
|
||
##Additional information |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,113 @@ | ||
# Chapter 1 | ||
|
||
|
||
## What is web archiving? | ||
|
||
Web archiving is the process of recording web resources. Various elements of the website such as HTML, scripts, images, videos, etc. can be recorded to preserve as much of the original resource as possible. By creating dynamic archives that provide a user with the same experience they would have if they accessed the original site, web archives can give much more information than a static screenshot. | ||
|
||
Web archives may be utilized by future researchers, historians and the general public. | ||
|
||
## Why Autopilot? | ||
|
||
Autopilot is a tool that navigates a website similarly to how a human would, doing things like scrolling, clicking buttons, and playing videos. These actions executed through functions called "behaviors." Because Autopilot can go through the technically complicated aspects of a website, it is useful for recording high-fidelity websites. Furthermore, Autopilot is designed to be accessible for anyone to use, and anyone with little javascript knowledge to contribute to. This makes webarchiving available to everyone. | ||
|
||
# Autopilot basics | ||
Autopilot uses [behaviors](https://github.com/webrecorder/behaviors/blob/master/manual/behaviors.md) to collect metadata from websites. Behaviors are Javascript modules which perform a series of actions on a webpage in order to collect information. | ||
|
||
## Installing Autopilot | ||
|
||
To use this project you must first install its dependencies. You can do this via a package manager like [Yarn](https://classic.yarnpkg.com/en/docs/install/#mac-stable) or [npm](https://www.npmjs.com/). | ||
|
||
``` | ||
$ yarn install | ||
# or "npm install" | ||
``` | ||
|
||
To install Autopilot, use Terminal to clone the Webrecorder repository: | ||
|
||
**You must be using node 12~** | ||
|
||
``` | ||
1. `git clone https://github.com/webrecorder/behaviors.git` | ||
2. `cd behaviors` | ||
3. `yarn build-watch` | ||
4. `open new terminal window` | ||
5. `bash scripts/init.sh` | ||
``` | ||
If you wish to use this project via Docker, install [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/). | ||
|
||
``` | ||
3. docker-compose build | ||
4. docker-compose up -d | ||
``` | ||
|
||
The Autopilot instance can be accessed in the browser at `http://localhost:8089`. | ||
|
||
# Running Behaviors | ||
1. In order to run the behaviors you will first need to build them: | ||
``` | ||
npm run generate-runnable-behaviors | ||
<!-- npm run build-dev --> | ||
``` | ||
|
||
2. Then, go into the dist directory. | ||
``` | ||
cd dist | ||
``` | ||
|
||
3. In the dist directory you will see behavior files. Open the behavior file you want and copy the code. | ||
``` | ||
open [filename] | ||
``` | ||
|
||
4. Copy all of the behavior code and paste it into the developer console of a webpage. The console can be accessed in Chrome using the shortcut: Option + ⌘ + J (on macOS), or Shift + CTRL + J (on Windows/Linux). | ||
|
||
You can also run behaviors using the CLI: | ||
|
||
## CLI and API | ||
|
||
The cli provides two commands `API` and `behaviors` and each command has its own options. | ||
|
||
## Behaviors command | ||
|
||
Execute `./bin/cli behaviors -b` to build the behaviors made available. This will build the behaviors using the behavior config file located in the root of this project. | ||
|
||
The built behaviors, along with a behavior metadata file (`behaviorMetadata.js`), can be found in the `dist` directory which will be created for you if it does not exist in the root of this project. | ||
|
||
## API command | ||
|
||
To run the behavior API server execute `./bin/cli api --build-behaviors`. | ||
|
||
This will start the API server after all behaviors provided by this project have been built. | ||
|
||
If you have already built the behaviors using the `behaviors` command provided by the cli then you may omit the `--build-behaviors` flag. | ||
|
||
|
||
Some configuration of the API server can be done via the environment variables listed below | ||
|
||
* `BEHAVIOR_API_HOST`: the host the api server will use (e.g. 127.0.0.1) | ||
* `BEHAVIOR_API_PORT`: the port the api server will listen on (e.g. 3030) | ||
* `WR_BEHAVIOR_DIR`: path to the directory containing the built behaviors | ||
* `WR_BEHAVIOR_METADATA_PATH`: path to the behavior metadata file | ||
* `BUILD_BEHAVIORS`: should the api server build the behaviors before starting | ||
|
||
|
||
|
||
# Docker | ||
If you would like, you can use Docker to run Autopilot. | ||
|
||
To build the wr-behaviors docker image (`webrecorder/behaviors:latest`) execute `docker-compose build`. | ||
|
||
The image created is suitable for building behaviors and running the behavior api server. | ||
|
||
The default configuration of the image is to run the api server, however you can substitute the default command with any of the cli commands listed previously. | ||
|
||
For more information please consult the provided `Dockerfile` and `docker-compose.yml` files. | ||
|
||
<!-- ## About Web Traffic | ||
--> | ||
|
||
<!-- ## Status page: pre-made behaviors | ||
blocked by website | ||
--> | ||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this could be in
.github/ISSUE_TEMPLATE/bug-report.md
with the following at the top of the file