diff --git a/README.md b/README.md index 40c303e5..7418cbd1 100644 --- a/README.md +++ b/README.md @@ -2,9 +2,9 @@ If you are interested in learning more about this repository, please speak with https://github.com/thesteve0 -The majority of the PostGIS Scenarios come from [this repository](https://github.com/postgis/postgis-workshops +The majority of the PostGIS Scenarios come from [this repository](https://github.com/postgis/postgis-workshops). License ------- -This content is licensed under the [Creative Commons Attribution-ShareAlike 4.0](https://creativecommons.org/licenses/by-sa/4.0/).) +This content is licensed under the [Creative Commons Attribution-ShareAlike 4.0](https://creativecommons.org/licenses/by-sa/4.0/). diff --git a/branded-ui/katacoda/templates/header.html b/branded-ui/katacoda/templates/header.html index 600956b7..cb9f470e 100644 --- a/branded-ui/katacoda/templates/header.html +++ b/branded-ui/katacoda/templates/header.html @@ -3,9 +3,15 @@ CrunchyData Interactive Learning Portal - - - + + + + + + + + + diff --git a/pg-administration/basic-postgresql-for-dbas/pgbackrest/10-pg-setup.md b/pg-administration/basic-postgresql-for-dbas/pgbackrest/10-pg-setup.md index a4f4c8c7..09904fb3 100644 --- a/pg-administration/basic-postgresql-for-dbas/pgbackrest/10-pg-setup.md +++ b/pg-administration/basic-postgresql-for-dbas/pgbackrest/10-pg-setup.md @@ -1,11 +1,11 @@ -`pg_basebackup` is included with PostgreSQL and is great for many filesystem backup scenarios. However, it can be lacking in many advanced features that enterprise backup solutions require; incrementals & differentials, retention management, parallelism, and more. pgBaseBackup is a third-party tool that aims to provide these advanced features and much more! +`pg_basebackup` is included with PostgreSQL and is great for many filesystem backup scenarios. However, it can be lacking in many advanced features that enterprise backup solutions require; incrementals & differentials, retention management, parallelism, and more. pgBackRest is a third-party tool that aims to provide these advanced features and much more! pgBackrest is available in the PGDG Yum repositories. These have already been set up in this scenario, so it's just a matter of installing the package ``` sudo yum install -y pgbackrest ```{{execute T1}} -Next some settings in the postgresql.conf must be updated to ensure WAL archiving is working. `archive_mode` must be turned on and the `archive_command` must be set to use backrest's `archive-push` command. A stanza name for the pgbackrest repo must also be given and that will be set up in the next step. Since we're changing these values on a running system, the ALTER SYSTEM command can be used +Next some settings in the postgresql.conf must be updated to ensure WAL archiving is working. `archive_mode` must be turned on and the `archive_command` must be set to use pgBackRest's `archive-push` command. A stanza name for the pgBackRest repo must also be given and that will be set up in the next step. Since we're changing these values on a running system, the ALTER SYSTEM command can be used ``` psql -c "ALTER SYSTEM SET archive_mode = 'on'" ```{{execute T1}} @@ -16,10 +16,6 @@ Archive mode isn't typically turned on by default and changing this setting requ ``` sudo systemctl restart postgresql-11 ```{{execute T1}} -You may see errors in the postgresql logs about the archive_command failing. This is normal and should resolve itself once the pgbackrest repository is set up. Do note that until the errors subside, the database will retain all WAL files generated in the `pg_wal` folder until it succeeds. In the case where the pgbackrest repo may not be available yet for an extended period of time, the easiest thing to do is to set `archive_command = '/bin/true'` until then. The command itself only requires a reload of the database to change, so that allows you to at least enable `archive_mode` until it's ready. - -Next is configuring pgbackrest itself. - - - +You may see errors in the postgresql logs about the archive_command failing. This is normal and should resolve itself once the pgBackRest repository is set up. Do note that until the errors subside, the database will retain all WAL files generated in the `pg_wal` folder until it succeeds. In the case where the pgBackRest repo may not be available for an extended period of time, the easiest thing to do is to set `archive_command = '/bin/true'` until then. The command itself only requires a reload of the database to change, so that allows you to at least enable `archive_mode` until it's ready. +Next is configuring pgBackRest itself. diff --git a/pg-administration/basic-postgresql-for-dbas/pgbackrest/20-backrest-setup.md b/pg-administration/basic-postgresql-for-dbas/pgbackrest/20-backrest-setup.md index c3ff3689..67ccd785 100644 --- a/pg-administration/basic-postgresql-for-dbas/pgbackrest/20-backrest-setup.md +++ b/pg-administration/basic-postgresql-for-dbas/pgbackrest/20-backrest-setup.md @@ -1,4 +1,4 @@ -A minimal `/etc/pgbackrest.conf` file is created by the package installation. We will be configuring a global section for settings that are common to all stanzas. And then some stanza specific settings. It's also possible to group settings into sections for specific commands as well. You can either run the command below or manually edit the config file yourself +A minimal `/etc/pgbackrest.conf` file is created by the package installation. We will be configuring a global section for settings that are common to all stanzas. And then some stanza specific settings. It's also possible to group settings into sections for specific commands as well. You can either run the command below or manually edit the config file yourself ``` sudo bash -c "cat > /etc/pgbackrest.conf << EOF [global] @@ -11,19 +11,16 @@ retention-full=2 EOF" ```{{execute T1}} -The `global` section has settings for the pgbackrest repository that will be common for any stanzas that are created. This is very useful when you have a dedicated backup system for many databases that should have some common settings such as the repository location and logging levels. The log level is also slightly increased from the default to give slightly more feedback when running commands manually. +The `global` section has settings for the pgBackRest repository that will be common for any stanzas that are created. This is very useful when you have a dedicated backup system for many databases that should have some common settings such as the repository location and logging levels. The log level is also slightly increased from the default to give slightly more feedback when running commands manually. -Each stanza's settings are then placed under a `[stanza-name]` heading. In this case, it contains the path to our database's data directory and our retenion policy. Here it is keeping 2 full backups. When the next ***successful*** backup exceeds this number, pgbackrest will automatically expire older backups. +Each stanza's settings are then placed under a `[stanza-name]` heading. In this case, it contains the path to our database's data directory and our retention policy. Here it is keeping 2 full backups. When the next ***successful*** backup exceeds this number, pgBackRest will automatically expire older backups. With the configuration in place, the stanza can now be created in our repository. You'll typically be wanting to run the backups as the postgres system user, not root. The archive_command is run as the postgres system user, so it needs to be able to write to the repository. And then this allows the backups to be run as the postgres user as well, without requiring root. ``` sudo -Hiu postgres pgbackrest --stanza=main stanza-create ```{{execute T1}} -It's then good to run the check command to ensure pgbackrest is configured and working properly +It's then good to run the check command to ensure pgBackRest is configured and working properly ``` sudo -Hiu postgres pgbackrest --stanza=main check ```{{execute T1}} -If either of these commands fail, you can check the pgbackrest logs located in `/var/log/pgbackrest` by default. If the issue is with the archive command, additional informaton as to why it's failing can also be found in the postgresql logs themselves (`/var/lib/pgsql/11/data/log`). - - - +If either of these commands fail, you can check the pgBackRest logs located in `/var/log/pgbackrest` by default. If the issue is with the archive command, additional information as to why it's failing can also be found in the postgresql logs themselves (`/var/lib/pgsql/11/data/log`). diff --git a/pg-administration/basic-postgresql-for-dbas/pgbackrest/30-running-backup.md b/pg-administration/basic-postgresql-for-dbas/pgbackrest/30-running-backup.md index c0a5fc6c..54612874 100644 --- a/pg-administration/basic-postgresql-for-dbas/pgbackrest/30-running-backup.md +++ b/pg-administration/basic-postgresql-for-dbas/pgbackrest/30-running-backup.md @@ -12,7 +12,3 @@ Run the above two commands a few more times. If you cause more than 2 fulls to b ``` sudo -Hiu postgres pgbackrest info ```{{execute T1}} - - - - diff --git a/pg-administration/basic-postgresql-for-dbas/pgbackrest/40-backrest-restore.md b/pg-administration/basic-postgresql-for-dbas/pgbackrest/40-backrest-restore.md index 76df19c3..760b5a13 100644 --- a/pg-administration/basic-postgresql-for-dbas/pgbackrest/40-backrest-restore.md +++ b/pg-administration/basic-postgresql-for-dbas/pgbackrest/40-backrest-restore.md @@ -1,6 +1,6 @@ -If we need to restore our backups at any time, say due to a dropped table, pgbackrest allows that to be done very efficiently. And with Point-In-Time recovery, we can restore the database to a specific time we know that it was in a good state. +If we need to restore our backups at any time, say due to a dropped table, pgBackRest allows that to be done very efficiently. And with Point-In-Time recovery, we can restore the database to a specific time when we know that it was in a good state. -We'll need to know the time before our table was dropped for this example, so we'll make note of that now using an environment variable to be used later. It is important to represent the time as reckoned by PostgreSQL and to include timezone offsets. This reduces the possibility of unintended timezone conversions and an unexpected recovery result. +We'll need to know the time before our table was dropped for this example, so we'll make note of that now using an environment variable to be used later. It is important to represent the time as reckoned by PostgreSQL and to include timezone offsets. This reduces the possibility of unintended timezone conversions and an unexpected recovery result. ``` export RESTORETIME=$(psql -Atc "select current_timestamp") echo $RESTORETIME @@ -15,7 +15,7 @@ We're now in disaster recovery mode, so we'll want to shut down our primary data ``` sudo systemctl stop postgresql-11 ```{{execute T1}} -Note that the last backup is always used by default when running the restore command. If the last backup happened after the desired point in time, the `--set` command can be given to pgbackrest to tell it to use an earlier backup that still exists in the repository. Also by default, the restore location must be empty. But if we're restoring to an existing cluster, the `--delta` option can be used to just overwrite what has changed since the last backup. This can GREATLY speed up recovery times. +Note that the last backup is always used by default when running the restore command. If the last backup happened after the desired point in time, the `--set` command can be given to pgBackRest to tell it to use an earlier backup that still exists in the repository. Also by default, the restore location must be empty. But if we're restoring to an existing cluster, the `--delta` option can be used to just overwrite what has changed since the last backup. This can GREATLY speed up recovery times. ``` sudo -Hiu postgres pgbackrest --stanza=main --delta --type=time "--target=$RESTORETIME" --target-action=promote restore ```{{execute T1}} @@ -39,6 +39,6 @@ sudo -Hiu postgres chmod 700 /var/lib/pgsql/11/br_restore ``` sudo -Hiu postgres pgbackrest restore --stanza=main --db-path=/var/lib/pgsql/11/br_restore ```{{execute T1}} -If this is done, it is critically important to ***TURN OFF THE ARCHIVE COMMAND***. Otherwise, if your target destination also has write access to the pgbackrest repository, you will have two locations writing there and likely corrupt the entire backup. As stated earlier, the easiest way to disable the archive_command on a linux system is to set it to `/bin/true`. +If this is done, it is critically important to ***TURN OFF THE ARCHIVE COMMAND***. Otherwise, if your target destination also has write access to the pgBackRest repository, you will have two locations writing there and likely corrupt the entire repository. As stated earlier, the easiest way to disable the archive_command on a linux system is to set it to `/bin/true`. -The Restore section of the pgbackrest userguide has much more thorough explanations of these restore situations - https://pgbackrest.org/user-guide-centos7.html#restore +The Restore section of the pgBackRest user guide has much more thorough explanations of these restore situations - https://pgbackrest.org/user-guide-centos7.html#restore diff --git a/pg-administration/basic-postgresql-for-dbas/pgbackrest/index.json b/pg-administration/basic-postgresql-for-dbas/pgbackrest/index.json index bef623fa..57fba466 100644 --- a/pg-administration/basic-postgresql-for-dbas/pgbackrest/index.json +++ b/pg-administration/basic-postgresql-for-dbas/pgbackrest/index.json @@ -1,14 +1,14 @@ { - "title": "PGBackRest", - "description": "PGBackRest", + "title": "pgBackRest", + "description": "pgBackRest", "difficulty": "beginner", "time": "20 minutes", "details": { "steps": [ - {"title": "Setting up PostgreSQL to use PGBackRest", "text": "10-pg-setup.md"}, - {"title": "Setting up PGBackrest", "text": "20-backrest-setup.md"}, + {"title": "Setting up PostgreSQL to use pgBackRest", "text": "10-pg-setup.md"}, + {"title": "Setting up pgBackRest", "text": "20-backrest-setup.md"}, {"title": "Running Backups", "text": "30-running-backup.md"}, - {"title": "Restoring with PGBackrest", "text": "40-backrest-restore.md"} + {"title": "Restoring with pgBackRest", "text": "40-backrest-restore.md"} ], "intro": { "courseData": "env-init.sh", diff --git a/pg-administration/basic-postgresql-for-dbas/pgbackrest/intro.md b/pg-administration/basic-postgresql-for-dbas/pgbackrest/intro.md index 8493ec28..17eec316 100644 --- a/pg-administration/basic-postgresql-for-dbas/pgbackrest/intro.md +++ b/pg-administration/basic-postgresql-for-dbas/pgbackrest/intro.md @@ -1,6 +1,6 @@ -# PGBackRest -This scenario will go over the basics of using pgbackrest to perform backups and restoration. It has many, many more options available within it for customizing your backup processes, but this scenario should help you get started with it. - +# pgBackRest +This scenario will go over the basics of using pgBackRest to perform backups and restoration. It has many, many more options available within it for customizing your backup processes, but this scenario should help you get started with it. + Full documentation is available at https://pgbackrest.org/ Let's get started! diff --git a/postgresql-devel/basics-pathway.json b/postgresql-devel/basics-pathway.json index a402f03a..9c5eed65 100644 --- a/postgresql-devel/basics-pathway.json +++ b/postgresql-devel/basics-pathway.json @@ -11,5 +11,11 @@ "course_id": "basicpgadmin", "title": "Using PgAdmin4 as a GUI to your PostgreSQL Server" } + , + { + "external_link": "https://learn.crunchydata.com/postgresql-devel/courses/basics/intropsql", + "course_id": "intropsql", + "title": "Introduction to Using the Command Line with PostgreSQL" + } ] } diff --git a/postgresql-devel/basics/intropsql/01-intro-psql.md b/postgresql-devel/basics/intropsql/01-intro-psql.md new file mode 100644 index 00000000..b0fc357e --- /dev/null +++ b/postgresql-devel/basics/intropsql/01-intro-psql.md @@ -0,0 +1,43 @@ +## Introduction to `psql` + +`psql` is described in the [docs](https://www.postgresql.org/docs/12/app-psql.html) as a "terminal-based front-end to PostgreSQL." This means that instead of having to use a GUI tool such as [pgAdmin](https://www.pgadmin.org/) to create, access, or manipulate a database, you can use text commands via a command line interface (CLI). + +We'll get started by connecting to a database. + +### Connect to the `workshop` database + +The database contains storm events data, which is public domain data from the National Weather Service. + +In your interactive terminal, you're currently logged in as root user, as indicated by the command line prompt: + +![Terminal prompt](./assets/01_-_command_line_prompt.PNG) + +> **Note** +> +> Depending on the system you're using and the shell (i.e. program you're in), +> the prompt might look a little different. You'll see this below when you're +> in the `psql` shell. + +We want to connect to the PostgreSQL database as **groot**, so we'll enter the `psql` command, accompanied by some flags: + +``` +psql -h localhost -U groot -d workshop +```{{execute}} + +Each command line flag or option begins with a `-` or `--`, and what follows after the space is the parameter you're passing in with the command. + +1. The `-h` option specifies the host server (e.g. `localhost`), or socket directory. +2. `-U` specifies the user you're connecting as (`groot`). +3. `-d` specifies the database you're connecting to (`workshop`). + +The CLI shows a response whenever a command is run. In this case, since you are logging in to a database as the user **groot**, the command line will then prompt you for the password (enter `password`). You won't see the password displayed as you type, but the terminal is recording your keystrokes. + +![Terminal prompt for password](./assets/01_-_password_prompt.PNG) + +Once you're logged in to the database server, the prompt in the terminal should change. The psql prompt indicates the database you're connected to, along with `=>`. + +![Postgres prompt](01_-_postgres_prompt.PNG) + +The beauty of the command line is that there are often shortcuts or tips and tricks. For example, you could shorten the command used above even further to: + +`psql -h localhost workshop groot` diff --git a/postgresql-devel/basics/intropsql/02-metacommands.md b/postgresql-devel/basics/intropsql/02-metacommands.md new file mode 100644 index 00000000..2e2c1747 --- /dev/null +++ b/postgresql-devel/basics/intropsql/02-metacommands.md @@ -0,0 +1,54 @@ +## Meta-commands + +The `psql` shell comes with meta-commands, which begin with an unquoted backslash. + +### Get help + +An example of one is `\?`, which will bring up help information about commands available in `psql`. + +`\?`{{execute}} + +> **Tip** +> Press the spacebar to scroll through the list, or press `q` any time to return > to the `psql` prompt. + +You should see the list of available meta-commands, including the one you just used. You'll see that the `\?` command can also include parameters that allow you to look up specific topics, e.g. `\? options` will bring up help information on psql options (flags). + +The `\h` command will bring up a list of SQL commands. + +`\h`{{execute}} + +And if you add the name of a command, you'll be shown more detailed information: + +`\h ALTER INDEX`{{execute}} + +### Describe a table + +`\d` with the name of a table will display table metadata, e.g. columns, data types, and other attributes. + +`\d se_details`{{execute}} + +`\d` used without any parameters will show a list of all available tables in the current database. If `+` is appended, you'll also see extended information such as each table's size on disk. + +`\d+`{{execute}} + +Give it a try with a table name as well, and see the difference: + +`\d+ se_details`{{execute}} + +There are more in the `\d` set of meta-commands that you can use. Examples of more common ones are `\dn` (all schemas), `\dv` (all available views), `\du` (all users), `\df` (all functions). + +## Options + +As you saw in the earlier step, the `psql` command can be run with options. + +One useful option is `--help`, which displays help information about `psql`, and then exits (i.e. you don't stay in the `psql` shell). + +Let's log out of PostgreSQL first (by entering `\q`, or typing `CTRL`+`d`), and then run `psql` again with `--help` this time. + +`\q`{{execute}} + +`psql --help`{{execute}} + + diff --git a/postgresql-devel/basics/intropsql/03-run-query.md b/postgresql-devel/basics/intropsql/03-run-query.md new file mode 100644 index 00000000..a879d677 --- /dev/null +++ b/postgresql-devel/basics/intropsql/03-run-query.md @@ -0,0 +1,23 @@ +## Running queries in `psql` + +Once you're in the `psql` shell (i.e. logged in), you can run SQL queries. + +Let's connect to the database again: + +``` +psql -h localhost -U groot -d workshop +```{{execute}} + +``` +SELECT cz_name FROM se_details LIMIT 5; +```{{execute}} + +Note that SQL statements need to end in a semicolon. The `psql` command or other command-line statements, however, do not. + +### Multiline queries + +You can press `ENTER` in the `psql` shell if you want the SQL statement to continue on a new line -- `psql` won't process the statement until you add the semicolon at the end of a line. + +![Multiline query in terminal](./assets/03_-_multiline.PNG) + +You'll see that the command line prompt starts with a `=>`, but changes to a `->` if the statement continues on to the next line (i.e. if you press enter without ending with a `;`). The prompt doesn't change back to `=>` until the statement ends and runs. diff --git a/postgresql-devel/basics/intropsql/04-formatting.md b/postgresql-devel/basics/intropsql/04-formatting.md new file mode 100644 index 00000000..0af9ff7e --- /dev/null +++ b/postgresql-devel/basics/intropsql/04-formatting.md @@ -0,0 +1,33 @@ +## Formatting results using `\pset` + +Sometimes, your query might return a long list of results. The following statement, for example, returns over 55,000 rows: + +``` +SELECT state, + month_name, + event_type +FROM se_details; +```{{execute}} + +Remember that you can press `q` any time to return to the `psql` prompt. + +You can use the `\pset` command to customize the way results are displayed. + +Instead of having to scroll through the entire result set, you can instead have the terminal take you automatically to the end: + +`\pset pager`{{execute}} + +Running this command without a value (i.e. `\pset pager on` or `off`) toggles the pager use on or off. Try executing the SELECT query above again. + +`\pset` also has other options you can use to format the results display. Try running `\pset border 0` and `\pset border 2` to see how each one changes the display +when executing the query above. + +``` +\pset border 0 +```{{execute}} + +``` +\pset border 2 +```{{execute}} + +Note: There is also an option to set these preferences in a [startup file](https://www.postgresql.org/docs/current/app-psql.html#id-1.9.4.18.10), which can be run each time `psql` starts. diff --git a/postgresql-devel/basics/intropsql/05-scripting.md b/postgresql-devel/basics/intropsql/05-scripting.md new file mode 100644 index 00000000..dbe57733 --- /dev/null +++ b/postgresql-devel/basics/intropsql/05-scripting.md @@ -0,0 +1,47 @@ +## Scripting + +Getting more comfortable with the command line and `psql` allows you to expedite and automate tasks. + +One simple example is to use a SQL script to quickly create a new table and add some values +to it. Again, while you could also run SQL scripts with tools like pgAdmin, the command line +lets you do so with just a few keystrokes. + +We've included a file `test_create_and_copy.sql` in the interactive environment that does the following when run: + +1. Create a new table `cp_se_details` with the same table structure as `se_details`, +and copy only data from the month of June. +2. Create a new table `new_test_table` and populate it with a few values. +3. Display the contents of `new_test_table`. + +In the `psql` shell, we can use the `\i` command to run the contents of the file: + +``` +\i test_create_and_copy.sql +```{{execute}} + +The result of the `SELECT` statement in the file shows that `new_test_table` was actually created and contains data, but for good measure, let's use `\d+` to see changes made on the database level: + +`\d+`{{execute}} + +In addition to `new_test_table`, you should also be seeing the `cp_se_details` table. + +You don't even need to be in the `psql` shell to execute from a file. Let's say we were working outside of the `psql` shell, doing some other work on the command line: + +`\q`{{execute}} + +We want to check the contents of the script first. We can use the `cat` command (UNIX and Linux) to +read the contents of the file and display them as output in the terminal. + +``` +cat test_create_and_copy.sql +```{{execute}} + +If we're happy with what we see, we can then run `psql` to execute the SQL in this file. Note that the following command will execute the contents of the file, and then exit (per `psql --help`). (We won't run it this time, since you'll encounter errors trying to create the same tables from running `\i+` above.) + +``` +psql -h localhost -U groot -d workshop -f test_create_and_copy.sql +``` + +> **Note** +> +> We won't cover this in the course, but `psql` can also be used in shell scripts so that you can combine `psql` with other commands. This means that you can do a lot of work (including a series of complex tasks) in an efficient way. diff --git a/postgresql-devel/basics/intropsql/assets/01_-_command_line_prompt.PNG b/postgresql-devel/basics/intropsql/assets/01_-_command_line_prompt.PNG new file mode 100644 index 00000000..4fe430b9 Binary files /dev/null and b/postgresql-devel/basics/intropsql/assets/01_-_command_line_prompt.PNG differ diff --git a/postgresql-devel/basics/intropsql/assets/01_-_password_prompt.PNG b/postgresql-devel/basics/intropsql/assets/01_-_password_prompt.PNG new file mode 100644 index 00000000..ea16a699 Binary files /dev/null and b/postgresql-devel/basics/intropsql/assets/01_-_password_prompt.PNG differ diff --git a/postgresql-devel/basics/intropsql/assets/01_-_postgres_prompt.PNG b/postgresql-devel/basics/intropsql/assets/01_-_postgres_prompt.PNG new file mode 100644 index 00000000..769743e6 Binary files /dev/null and b/postgresql-devel/basics/intropsql/assets/01_-_postgres_prompt.PNG differ diff --git a/postgresql-devel/basics/intropsql/assets/03_-_multiline.PNG b/postgresql-devel/basics/intropsql/assets/03_-_multiline.PNG new file mode 100644 index 00000000..61aaaea4 Binary files /dev/null and b/postgresql-devel/basics/intropsql/assets/03_-_multiline.PNG differ diff --git a/postgresql-devel/basics/intropsql/env-init.sh b/postgresql-devel/basics/intropsql/env-init.sh new file mode 100644 index 00000000..84ce5ee8 --- /dev/null +++ b/postgresql-devel/basics/intropsql/env-init.sh @@ -0,0 +1,3 @@ +#!/usr/bin/bash + +# nothing for now diff --git a/postgresql-devel/basics/intropsql/finish.md b/postgresql-devel/basics/intropsql/finish.md new file mode 100644 index 00000000..85d3ddaf --- /dev/null +++ b/postgresql-devel/basics/intropsql/finish.md @@ -0,0 +1,9 @@ +## Final notes + +The command line might seem daunting, but even an understanding of some basic commands can increase your confidence as a developer and help you become more effective in your work. + +You've now gotten a taste of `psql` and . Here are a couple of resources we recommend for learning more: + +- The official [PostgreSQL documentation](https://www.postgresql.org/docs/current/app-psql.html) +- Common `psql` commands on [postgresqltutorial.com](https://www.postgresqltutorial.com/psql-commands/) +- [Linux Journey: Command Line](https://linuxjourney.com/lesson/the-shell) \ No newline at end of file diff --git a/postgresql-devel/basics/intropsql/index.json b/postgresql-devel/basics/intropsql/index.json new file mode 100644 index 00000000..b6baf354 --- /dev/null +++ b/postgresql-devel/basics/intropsql/index.json @@ -0,0 +1,49 @@ +{ + "title": "Introduction to Using the Command Line with PostgreSQL", + "description": "A quick intro to working with Postgres using the command line interface", + "difficulty": "beginner", + "time": "10 minutes", + "details": { + "steps": [ + { + "title": "The psql Command", + "text": "01-intro-psql.md" + }, + { + "title": "Psql Meta-commands and Options", + "text": "02-metacommands.md" + }, + { + "title": "Psql: Running Queries", + "text": "03-run-query.md" + }, + { + "title": "Psql: Query Results", + "text": "04-formatting.md" + }, + { + "title": "Psql and Scripting", + "text": "05-scripting.md" + } + ], + "intro": { + "courseData": "env-init.sh", + "code": "set-env.sh", + "text": "intro.md", + "credits": "" + }, + "finish": { + "text": "finish.md" + } + }, + "environment": { + "uilayout": "terminal", + "uimessage1": "\u001b[32mYour Interactive Bash Terminal.\u001b[m\r\n", + "terminals": [ + {"name": "Terminal 2", "target": "host01"} + ] + }, + "backend": { + "imageid": "crunchydata-single1" + } +} diff --git a/postgresql-devel/basics/intropsql/intro.md b/postgresql-devel/basics/intropsql/intro.md new file mode 100644 index 00000000..2b1b0de8 --- /dev/null +++ b/postgresql-devel/basics/intropsql/intro.md @@ -0,0 +1,10 @@ +# Introduction to Using the Command Line with PostgreSQL + +`psql` is a command line tool that is included with a PostgreSQL install. It lets you run queries and scripts, manage databases, and interact with the computer using a command-line client. + +The advantage of using the command line is that it can save you time - entering short text commands might be even faster than clicking through a graphical user interface (GUI), especially as you become more and more comfortable with the command line. Additionally, there are some functions that you can perform only in the command line. You can even use psql within a shell script, allowing you to execute multiple commands together and automate repetitive tasks. + +In this scenario, you'll use psql to interact with a PostgreSQL database. A database with pre-loaded data has already been added for you. You'll use the following credentials to connect to the `workshop` database: + +1. Username: groot +2. Password: password \ No newline at end of file diff --git a/postgresql-devel/basics/intropsql/set-env.sh b/postgresql-devel/basics/intropsql/set-env.sh new file mode 100644 index 00000000..5afec8ff --- /dev/null +++ b/postgresql-devel/basics/intropsql/set-env.sh @@ -0,0 +1,46 @@ +#!/usr/bin/bash + +echo 'please wait while we prep the environment (should take about 10 seconds)' +echo 'starting the database' +docker network create mybridge +docker run -d --network mybridge -p 5432:5432 -e PG_USER=groot -e PG_PASSWORD=password -e PG_DATABASE=workshop --name=pgsql thesteve0/postgres-appdev + +until PGPASSWORD="password" psql -h localhost -U groot postgres -c '\l' &> /dev/null; do + echo >&2 "$(date +%Y%m%dt%H%M%S) Waiting for Postgres to start" + sleep 1 +done + +echo 'loading storm center points' +PGPASSWORD="password" psql -h localhost -U groot -f /data/crunchy_demo_data/storms/stormevents.ddl.sql workshop +PGPASSWORD="password" psql -h localhost -U groot -d workshop -c '\COPY se_details from '\''/data/crunchy_demo_data/storms/StormEvents_details-ftp_v1.0_d2018_c20190130.csv'\'' WITH CSV HEADER' + +echo 'finished storm center points' + +cat < test_create_and_copy.sql +-- Partial copy of se_details table +CREATE TABLE cp_se_details AS +SELECT + * +FROM se_details +WHERE month_name = 'June'; + +-- Create another table and insert some data +CREATE TABLE new_test_table ( + id SERIAL PRIMARY KEY, + hero_name VARCHAR (80), + cohort integer +); +INSERT INTO new_test_table (hero_name, cohort) +VALUES ('Iron Man', 1), + ('Captain Marvel', 3), + ('Incredible Hulk', 1), + ('Black Panther', 2), + ('Black Widow', 3); + +-- Display contents of new table +SELECT * FROM new_test_table; +EOF + +clear + +: 'ready to go!' \ No newline at end of file