Skip to content

Latest commit

 

History

History
176 lines (125 loc) · 4.81 KB

README.md

File metadata and controls

176 lines (125 loc) · 4.81 KB
Raw

tap-playstore [Google Play (GCS Export)]

tap-playstore is a Singer tap to extract Google Play Store Console Reports that are available on google cloud storage.

The following streams can be integrated using this tap:

  • stats_by_dimension_buyers_7d
  • stats_by_dimension_crashes
  • stats_by_dimension_gcm
  • stats_by_dimension_installs
  • stats_by_dimension_ratings
  • stats_by_dimension_retained_installers
  • stats_overview_crashes
  • stats_overview_gcm
  • stats_overview_installs
  • stats_overview_ratings
  • reviews
  • store_performance_country
  • store_performance_traffic_source
  • subscriptions_country
  • earnings
  • play_balance_krw
  • sales_reports

note: the streams prefixed with stats_by_dimension consolidate all reports by dimension for the given metric into one common stream.

Built with the Meltano Tap SDK for Singer Taps.

Project Home

https://github.com/haleemur/tap-playstore

The project's source can be viewed on github

Installation

Install from PyPi:

pipx install tap-playstore

Install from GitHub:

pipx install git+https://github.com/haleemur/tap-playstore.git@main

Capabilities

  • catalog
  • state
  • discover
  • about
  • stream-maps
  • batch

Configuration

Accepted Config Options

Setting Required Default Description
service_account_json_file False None Google Cloud Service Account JSON file
service_account_json_str False None Google Cloud Service Account JSON string
start_date True None The earliest record date to sync
bucket_name True None The GCS Bucket where Play Console Reports are stored.
stream_maps False None Config object for stream maps capability. For more information check out Stream Maps.
stream_map_config False None User-defined config values to be used within map expressions.

NOTE: At least one of [service_account_json_file, service_account_json_str] are required.

A full list of supported settings and capabilities for this tap is available by running:

tap-playstore --about

Configure using environment variables

This Singer tap will automatically import any environment variables within the working directory's .env if the --config=ENV is provided, such that config values will be considered if a matching environment variable is set either in the terminal context or in the .env file.

Usage

You can easily run tap-playstore by itself or in a pipeline using Meltano.

Executing the Tap Directly

tap-playstore --version
tap-playstore --help
tap-playstore --config CONFIG --discover > ./catalog.json

Developer Resources

Follow these instructions to contribute to this project.

Initialize your Development Environment

pipx install poetry
poetry install
poetry run pre-commit install

Create and Run Tests

Tests require a test_config.json file to be present. Currently, running tests require an active connection to google cloud.

TODO: write proper mocks for google cloud storage.

The test_config.json file should mimic the structure of sample_config.json file.

{
    "start_date": "2024-01-01",
    "bucket_name": "pubsite_prod_rev_00000000000123456789",
    "service_account_json_file": "test_credentials.json"
}

Create tests within the tests subfolder and then run:

poetry run pytest

You can also test the tap-playstore CLI interface directly using poetry run:

poetry run tap-playstore --help

Testing with Meltano

Note: This tap will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.

Next, install Meltano (if you haven't already) and any needed plugins:

# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd tap-playstore
meltano install

Update the meltano config (or populate the environment variables / modify your .env file)

    config:
      start_date: '2024-01-01T00:00:00Z'
      service_account_json_file: $GOOGLE_PLAY_CREDENTIALS_FILE
      bucket_name: $GOOGLE_PLAY_BUCKET

Now you can test and orchestrate using Meltano:

# Test invocation:
meltano invoke tap-playstore --version
# OR run a test `elt` pipeline:
meltano elt tap-playstore target-jsonl

SDK Dev Guide

See the dev guide for more instructions on how to use the SDK to develop your own taps and targets.