Skip to content

Commit

Permalink
Squashed code for initial open-source GitHub release
Browse files Browse the repository at this point in the history
Original ref: a0867e69deb41b0731d1a42da3fa20ca9c9ebf56
  • Loading branch information
pinkeen committed Feb 4, 2020
0 parents commit ce8c653
Show file tree
Hide file tree
Showing 551 changed files with 16,437 additions and 0 deletions.
21 changes: 21 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
/tmp/*
!/tmp/.gitkeep

/artifacts/
/roles/*
!/roles/cs.*

/vars/*
/vars/global/*
/vars/project/*
!/var/global/.gitkeep
!/var/project/.gitkeep

*.pyc
__pycache__/
*.retry
*.box
*.tmp
.tmp.*

/test.*
25 changes: 25 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
MIT LICENSE

Copyright 2016-2020 creativestyle GmbH

https://creativestyle.de
Creativestyle GmbH
Erika-Mann-Straße 53
80636 München
Deutschland

Permission is hereby granted, free of charge, to any person obtaining a copy of this software
and associated documentation files (the "Software"), to deal in the Software without restriction,
including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial
portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT
LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

87 changes: 87 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
<p align="center">
<img alt="MageSuite" width="211" src="https://avatars2.githubusercontent.com/u/56443641?s=200&v=4">
</p>

<p align="center">
<em>Ansible-based infrastructure-as-code for hosting and deploying Magento-based shops.</em>
</p>

# Infrastructure Provisioning

## MageSuite ecosystem

Altough it's possible to use this code to hosting any Magento based shop,
some features are only avaialble (and tailored for) [MageSuite](https://magesuite.io/) ecosystem.

## Cloud infrastructre

Currently the full feature set is only available when using AWS as the cloud platform, however,
it's possible (and tested) to use this code to provision any CentOS 7 VM including local development environment.

_PS There's also an [experimental docker container](https://github.com/mageops/docker-centos-systemd) which can
be fully provisioned by this code and run a full CentOS installation like a VM. This is intended to be used
only for local development though and not yet supported_.

## Feature highlights

- Out-of-the-box autoscaling
- Zero-downtime deployments<sup>1</sup>
- Nodes are warmed up when new code is being deployed before being attached
to the varnish load-balancer to avoid cache generation penalty after traffic hit
- Varnish caching tailored for Magento with multiple configurable custom features
- Redis Magento cache backend with
- Separate redis instance for session storage
- Customizable multi-website / multi-shop deployments
- Letsencrypt certificates support
- Cloudfront CDN
- Media stored on S3
- Seamless image optimization via Lambda@Edge (with webp support)
- Support for file sharing across nodes via AWS EFS
- Per path pattern customizable PHP limits
- Customizable request throttling (rate-limiting)
- On-demand import instance
- EC2 instance started for scheduled imports automatically and terminated
once the import is finished
- Allows to use a different instance type for heavy scheduled imports
and perform them without disrupting the site performance for end-users
- Customizable rules for routing traffic to separate set of special purpose extra-instances
- Routing rules based on user-agent and request path patterns
- Useful for processing heavy workloads without disrupting shop performance (e.g. ERP Magento API integrations)
- Magento queue consumers support with RabbitMQ and supervisord
- Logs pushed to CloudWatch with slack notifications for critical errors
- VPC and SG-based network communication rules allowing internal service communication
strictly on a need-to basis
- ... any many more!

<sup>1</sup> Needs at least 2 app nodes and requires certain MageSuite extensions. Also it might not be
possible depending on changes in relation to previous release in the code being deployed (DB migrations, ...).

## Initial open-source release

The code has been cleaned up and all commits squashed for the first public release.

The previous internal repository had **2000+ commits**, **300+ merge requests** created by
10+ contributors across **3+ years of development**.

The current state is a result of hundreds of hours of meetings, debugging, brainstorms
investigations and combined expertise of all our developers, PMs and clients.

A lot of the solutions are tailored for solving real-world problems and use-cases that came
up when hosting and maintaining highly-customized Magento-based shops. The value of such
experience-based solutions cannot be overestimated. It wouldn't be possible to come up and
design all of these tiny details form scratch without having went through the all of problems first-hand.

## Documentation

At the time of open source release most of the documentation was scattered
across internal creativestyle services thus any leftovers have been removed
to avoid confusion. New documentation will start appearing shortly as part
of this repository succesively.


<p align="right">
<em>Brought to life by</em><br/>
<a href="https://creativestyle.pl">
<img src="https://images.squarespace-cdn.com/content/59144053ff7c509585045a38/1494845103833-S34MR8ONLUWAXOKQSM7O/CS_logo_colour.png?format=500w&content-type=image%2Fpng" width="160"/>
</a>
</p>
50 changes: 50 additions & 0 deletions ansible.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
[defaults]
ansible_python_interpreter = python

remote_user = root
host_key_checking = no

retry_files_enabled = no

forks = 20
internal_poll_interval = 0.001

stdout_callback = yaml
deprecation_warnings = yes
show_custom_stats = yes
display_failed_stderr = yes
display_ok_hosts = yes
display_skipped_hosts = yes

bin_ansible_callbacks = yes

# fact_caching = yaml
fact_caching_connection = tmp/fact_cache
fact_caching_timeout = 2700
fact_caching_prefix = mageops-facts

callback_whitelist = timer, profile_tasks

[callback_profile_tasks]
task_output_limit = 50
order = descending

[inventory]
enable_plugins = host_list, script, auto, yaml, ini, aws_ec2, constructed
cache = no
cache_plugin = yaml
cache_connection = tmp/inventory_cache
cache_timeout = 1800
cache_prefix = mageops-inventory


[ssh_connection]
pipelining = yes
control_path_dir = ~/.local/var/ssh
control_path = %(directory)s/%%C
ssh_args = -o ControlMaster=auto -o ControlPersist=30m -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null

[persistent_connection]
command_timeout = 30
connect_timeout = 30
connect_retry_timeout = 30
Binary file added docs/assets/aws-diagram.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/creativestyle.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/mageops.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file added filter_plugins/__init__.py
Empty file.
12 changes: 12 additions & 0 deletions filter_plugins/sanitize.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type


from ansible.plugins.inventory import to_safe_group_name

class FilterModule(object):
def filters(self):
return {
'to_safe_group_name': to_safe_group_name,
}

Loading

0 comments on commit ce8c653

Please sign in to comment.