add mkdocs guide

.gitignore

@@ -1,3 +1,4 @@
 secrets.php
 proxy/file_cache
 debug.log
+.idea

docs/configuration.md

@@ -0,0 +1,172 @@
+# How to configure CiviProxy
+!!! caution
+    We assume you have a very basic understanding of PHP and editing PHP files. If you do not, it is probably a good idea to get some support from a CiviCRM expert. You can find one on [Find an Expert](https://civicrm.org/partners-contributors). 
+## Configuration of the CiviCRM extension
+You can use CiviProxy to ensure that:
+
+* you can still see all the stuff on the CiviMailing report like the click-throughs and bounces and such;
+* you do not want the links in your bulk mailing or individual mails to immediately feed back to your CiviCRM installation but pass through CiviProxy
+* you want your subscribe/unsubscribe links to pass through CiviProxy rather than go directly to your CiviCRM installation
+
+If you want to do any of these things, you will need to install and configure the CiviCRM extension **de.systopia.civiproxy** in your CiviCRM installation.
+
+!!! note
+    If you do not install the **de.systopia.civiproxy** extension you can still use CiviProxy to whitelist your API requests.
+    
+Once you have installed the CiviCRM extension (check the [Installing CiviProxy](installation.md) section of this guide for instructions) you will need to configure the CiviProxy settings. To do this, go to Administer/Administration Console. You will see the CiviProxy Settings in the System Settings section of the menu as you can see below.
+
+![Administration Console - System Settings](img/administration_console.png)
+
+If you click on the CiviProxy Settings you will get a page with a few settings for CiviProxy. If you access the page for the first time these will be empty. The settings you can enter are:
+
+1. A checkbox to enable or disable the CiviProxy functionality with your CiviCRM mailing links.
+1. A text field where you can add the URL of your CiviProxy server. This link will be validated and the version of the CiviProxy will be shown. If the URL does not meet the validation (does not contain a valid CiviProxy installation), there will be an error message.
+1. A text field for a path to the unsubscribe page that will be used. There is a basic unsubscribe page on your CiviProxy server, the path to this page will be the default. If the default unsubscribe page is not what you would like, then you can either update the page in the CiviProxy installation on your server or create your own page and set the URL here.
+
+!!! note
+    If you do create your own subscription page make sure the parameters and returns and such are the same as in the default unsubscribe page!
+
+In the screen print below you can see the CiviProxy Settings page with the values I have used for my test installation when producing this guide.
+
+![Example CiviProxy Settings](img/civicrm_proxy_settings.png)
+## Configuration of the CiviProxy server
+The CiviProxy server is the actual policeman that receives all requests and decides if they are allowed to send data to or retrieve data from CiviCRM. It consists of a series of scripts which you need to install on the server as explained in [Installing CiviProxy](installation.md).
+
+Once you have installed your CiviProxy server you need to complete a few configuration steps.
+### The Config.php file
+The configuration of CiviProxy is mainly controlled with one PHP file called `config.php`. You will need to locate that file in your CiviProxy scripts:
+
+![List of files on your CiviProxy server](img/file%20list%20proxy.png)
+### Configuring the URL of your CiviProxy server
+First thing you need to configure is the base URL of your CiviProxy server using the `$proxy_base` variable in the `config.php` file:
+```php
+ // this should point to the base address of the CiviProxy installation
+ $proxy_base     = 'http://localhost/proxy';
+```
+### Configuring the link to the secure target CiviCRM
+Next thing you want to configure is what your target CiviCRM is. This is the CiviCRM installation which you want CiviProxy to police, so the one where the actual data resides and is collected from or sent to. The assumption is that this CiviCRM resides in some kind of VPN and will accept traffic only from the CiviProxy IP address (and probably a few trusted others like home workers or support people).
+You can set the URL of the target CiviCRM using the variable `$target_civirm` in the `config.php` file:
+```php
+ // this should point to the target CiviCRM system
+ $target_civicrm = 'http://localhost/ehdev';
+```
+### Setting for the click tracking url
+There is a setting in CiviCRM which is used for tracking the clicks in your mailing. On your CiviProxy server this setting is captured in the variable `$target_url` in the `config.php` file:
+```php
+$target_url       = $target_civicrm . '/sites/all/modules/civicrm/extern/url.php';
+```
+If you set it to the value NULL this functionality will not be available on your CiviProxy server.
+### Setting for the open tracking
+CiviCRM tracks if a mailing has been opened in a certain way. CiviProxy has this setting in the variable `$target_open` in the `config.php` file:
+```php
+$target_open      = $target_civicrm . '/sites/all/modules/civicrm/extern/open.php';
+```
+If you set it to the value NULL this functionality will not be available on your CiviProxy server.
+### Setting for the location of images and included files in your mail(ing)
+CiviCRM stores images and attachments you include in your (bulk) mail in a specific folder. In CiviProxy the name of this folder is stored in variable `$target_file` in the `config.php` file:
+```php
+$target_file      = $target_civicrm . '/sites/default/files/civicrm/persist/';
+```
+If you set it to the value NULL this functionality will not be available on your CiviProxy server.
+
+!!! note
+    By default CiviProxy will cache the files so it does not have to file from CiviCRM for each individual mail that is part of a bulk mailing. The default settings can be found in the `config.php` file:
+    ```php
+    /****************************************************************
+     **                   File Caching Options                     **
+     ****************************************************************/
+    // define file cache options, see http://pear.php.net/manual/en/package.caching.cache-lite.cache-lite.cache-lite.php
+    $file_cache_options = array(
+        'cacheDir' => 'file_cache/',
+        'lifeTime' => 86400
+    );
+    // define regex patterns that shoud NOT be accepted
+    $file_cache_exclude = array();
+    // if set, cached file must match at least one of these regex patterns
+    $file_cache_include = array(
+            //'#.+[.](png|jpe?g|gif)#i'           // only media files
+    );
+    ```
+    You can update these settings if you should want to.
+    
+    You can find documentation on the caching engine used in CiviProxy [here](http://pear.php.net/manual/en/package.caching.cache-lite.cache-lite.cache-lite.php)
+### Setting for the view unformatted mail link
+If CiviCRM sends a mail you can get a link to view the mail unformatted. CiviProxy keeps this setting in a variable `$target_mail_view` in the `config.php` file:
+```php
+$target_mail_view = $target_civicrm . '/civicrm/mailing/view';
+```
+If you set it to the value NULL this functionality will not be available on your CiviProxy server.
+### Setting for the REST API requests
+When you want to use the CiviCRM API from another server than the one CiviCRM itself resides on you will do so using the REST interface.
+
+!!! seealso
+    Generic information on CiviCRM with REST [here](https://wiki.civicrm.org/confluence/display/CRMDOC/REST+interface)
+
+CiviProxy has a setting which specifies where the REST script is located in the variable `$target_rest` in the `config.php` file:
+```php
+$target_rest      = $target_civicrm . '/sites/all/modules/civicrm/extern/rest.php';
+```
+If you set it to the value NULL this functionality will not be available on your CiviProxy server.
+### Configuring the API key for (un)subscribe
+CiviCRM allows subscribe/unsubscribe links in mailings. To process these a user needs to be configured with a specific API key. In CiviProxy this API key is held in the `config.php` file in variable ``:
+```php
+$mail_subscription_user_key = 123teSt!ExaMpl1;
+```
+If you set it to the value NULL this functionality will not be available on your CiviProxy server.
+### Configuring the API and Site key
+To be able to access your target CiviCRM with the API using REST you will need to configure an API key and a site key. Actually, you will most likely need more to be able if more outside applications are allowed to access your target CiviCRM.
+
+!!! seealso
+    Generic information on CiviCRM with REST [here](https://wiki.civicrm.org/confluence/display/CRMDOC/REST+interface)
+
+In CiviProxy you can either store the API and Site keys directly in the `config.php` file:
+```php
+$api_key_map = array();
+$sys_key_map = array();
+```
+or you can include a `secrets.php` file in your installation which holds your keys:
+```php
+if (file_exists(dirname(__FILE__)."/secrets.php")) {
+  // keys can also be stored in 'secrets.php'
+  require "secrets.php";
+}
+```
+### Whitelisting API requests
+Even if you have entered your API and Site key, and the setting for the target REST is ok, you will not be able to use any API calls through CiviProxy just yet. As CiviProxy uses the **whitelisting** principle by default **no** API calls are allowed unless they have been whilelisted.
+
+Also **parameter sanitation** is used. This means that only the specified parameters are allowed, and only content data of the specified type will be allowed.
+
+You can whitelist an API request in the `config.php` file by populating the `$rest_allowed_actions` array:
+```php
+$rest_allowed_actions = array(
+  // this is an example:
+  'Contact' => array(
+      'getsingle' => array(
+        'email' => 'string',
+      ),
+    ),
+  );
+```
+The example above allows using the `Contact Getsingle` API request, and will only accept the parameter `email` which will have to hold data of the type `string`. 
+
+So basically it only allows retrieving data of a single contact at a time using the email to identify the single contact.
+### Debug setting
+CiviProxy has a `$debug` setting which allows you to add the name of a text file where all requests are send to. This can be used during the initial testing to see if everything is processed correcty. 
+```php
+// CAREFUL: only enable temporarily on debug systems. Will log all queries to given PUBLIC file
+$debug                      = NULL; //'debug.log';
+```
+If you set the `$debug` setting to NULL this feature will not be used.
+
+!!! warning
+    We recommend only using this debug feature in the testing face, never on an active production installation as it can produce a massive log file quite easily.
+### Target Interface setting
+There is a setting for a local network interface or IP to be used for relayed queries. If you have no idea what this is about, just leave it as is.
+```php
+// Local network interface or IP to be used for the relayed query 
+// This is usefull in some VPN configurations (see CURLOPT_INTERFACE)
+$target_interface           = NULL;
+```
+
+!!! tip
+    We do advice you to monitor what request are still being fired directly to your target CiviCRM once you installed and configured CiviProxy, and then find solutions for the situations if you still see some undesired access requests.

docs/enhancements.md

@@ -0,0 +1,12 @@
+# Future enhancements for CiviProxy
+The one future enhancement we have identified is the ability to whitelist API requests for a certain IP address. 
+
+Right now if we whitelist for example the `contact getsingle` API with the parameters `email`, all IP addresses accessing CiviProxy would be able to use that API request.
+
+The desired future feature would be to be able to whitelist an API call for a specific IP address. 
+For example the `Contact getsingle` with the parameter `email` is only whitelisted for IP address 123.45.678.1, and the `Contact getsingle` with the parameters `first_name` and `last_name` are whitelisted for IP address 123.45.678.2.
+
+The enhancement is registered [here](https://github.com/systopia/CiviProxy/issues/12)
+
+!!! tip
+    If you want to report bugs or suggest future enhancements please do so on the [GitHub repository](https://github.com/systopia/CiviProxy/issues).

docs/index.md

@@ -0,0 +1,65 @@
+# Introduction
+
+## Public facing website and CiviCRM
+In the CiviCRM world it is a fairly typical requirement to want your public facing website and CiviCRM to communicate with each other.
+You would probably want to do stuff like:
+
+* allow website visitors to sign a petition, make a donation or register for an event
+* allow registered website visitors to view and perhaps update their own data
+* get website visitors to sign up (or unsubscribe) for newsletters.
+
+However, for security and maintainability purposes it is recommended that you separate your CiviCRM installation and your public facing website. Factors that might influence this:
+
+- your biggest security risk is probably your public facing website where you want the whole world to be able to check, browse and do stuff, even without identifying themselves.
+- the data you want to protect the most is quite likely to be part of CiviCRM (data on your donors, peope who sign petitions, emailaddresses etc.).
+- you will need to install security upgrades on your website regularly as this is exposed to the public. However, you do not want to test all your business processes with each upgrade (which you will need to do if you have both website and CiviCRM on the same server to be sure they continue to work together).
+So we think it is a sound practise to separate CiviCRM and your public facing website, and mainly use the CiviCRM API to send data to or get data from CiviCRM. 
+However, we do not want every authorized user to be able to use all the API possibilities. And we also do not want to allow any user access to the CiviCRM API.
+
+And that is where **CiviProxy** comes in!
+
+## How does it work?
+The basic idea is to first put your CiviCRM (with probably a dummy CMS which you could use for your intranet) server into a VPN. This makes the server virtually inaccessible from the internet, and your users or your whole office will access it via a secure connection.
+
+But what about your public web pages, donation pages, data exchange with other systems, API request? This is where CiviProxy comes in: You get a small, secure, extra server and give it access to both, the internet and your VPN. It will act as a safe gateway for selected features of your CiviCRM that you would like to expose.
+
+In an example picture:
+
+![a picture of an example network architecture](img/network.jpg)
+    
+The advantages in a nutshell :thumbsup:
+
+* CiviProxy is CiviCRM specific, so it kind of understands how CiviCRM works. It is **designed** to work with CiviCRM in a fairly simple way.
+* CiviProxy uses **whitelisting**. This means it starts with the principle _nothing is allowed_ and then lets you configure what is allowed, rather than the other way around.
+* CiviProxy uses **parameter sanitation**, meaning that only named parameters are allowed, and then only with the allowed content type (string, integer etc.)
+
+
+
+## What can it do?
+Currently CiviProxy can expose/relay the following CiviCRM functions
+
+1. Serve resources for newsletters and mailings
+1. Cache those resources, taking load off your CiviCRM server
+1. Pass-through of tracking data on opening and click-through rates
+1. Sign-on and off of your newsletter (Webpage templates)
+1. Relay of whitelisted REST API calls for data exchange with other systems
+1. Perform input sanitation and parameter whitelisting for the REST API calls
+
+!!! attention
+    This software has not (yet) been audited for security.
+
+## Why not an application firewall?
+The traditional approach to this problem would be an application firewall / reverse proxy setup. However, CiviCRM can have very complex interactions with other systems (e.g. via the API), and a malicious request can sometimes only be detected by understanding the meaning of the individual parameters.
+
+Creating firewall rules for this level of detail is very complex and is very hard to maintain.
+
+For this reason we wanted to take another approach and build a simple "bridgehead" system that *understands* CiviCRM, thus making its configuration and maintenance as easy as possible.
+
+## Contents of this guide
+In this guide you will find pages on:
+
+* [technical requirements for CiviProxy](requirements.md)
+* [how to install CiviProxy](installation.md)
+* [how to configure CiviProxy](configuration.md)
+* [what to do if an outside application wants to communicate with CiviProxy](outside.md)
+* [future enhancements for CiviProxy](enhancements.md)

docs/installation.md

@@ -0,0 +1,46 @@
+# Installation
+
+CiviProxy has to run on its own server (with its own IP address) for maximum effect.
+
+The best option for a server on which you can install CiviProxy is a simple managed webspace, that you can rent cheaply from the hoster of your choice. It should have its own IP address, but other than that there are virtually no restrictions. This approach comes with a lot of advantages:
+ 
+* Since there are a multitude of clients like you on such a server, the level of security can be expected to be very high, and it should be managed and monitored by professionals 24/7.
+* For the same reason, the bandwith and connectivity of such a server should be very high as well. 
+* The price should not have to be huge
+
+!!! note
+    Obviously there is nothing stopping you from installing CiviProxy on a server you manage yourself, but that then automatically means you have to ensure you maintain a high level of security and update the stuff regularly!
+    
+Installing CiviProxy should be pretty straightforward:
+
+* Download the GitHub repository from [https://github.com/systopia/CiviProxy]()
+* In your repository are actually two relevant parts:
+
+    * a CiviCRM extension called **de.systopia.civiproxy** that you can use to send mailings (newsletters) from CiviCRM. This should be installed on your target CiviCRM installation.
+    * a **proxy** folder with the scripts that you need to install on your CiviProxy server.
+    * On top of that there will also be a **docs** folder containing this guide and a few necessary files (which are needed but you can ignore):
+        * LIBRARIES.md
+        * LICENSE
+        * README.md
+        * mkdocs.yml
+ 
+## Installing the CiviCRM extension on your target CiviCRM
+* copy the folder **de.systopia.civiproxy** that you downloaded in the previous step into your CiviCRM extensions folder. You should be able to find your civicrm extensions folder in Administer/System Settings/Directories. If you have never touched or changed this it will look like screen print below. In that case your folder is probably `<your civicrm folder>sites/default/files/civicrm/ext/`
+
+!!! seealso
+    ![Screen print of the Administer/System Settings/Directories form](img/civicrm_directories.png)
+     
+* go to Administer/System Settings/Extensions to see a page with all the installed extensions on your CiviCRM installations. You will have to click the **Refresh** button to see the **de.systopia.civiproxy** extension. Click the **install** action listed behind the extension information. Upon successfull installation you should see a list like the one below (although you will probably see a larger list as you will have more extensions installed).
+
+!!! seealso
+    ![Screen print of the extensions page after successfull installation](img/extensions_list.png)
+    
+* type the URL (if you have enabled CleanUrls or `<your website>?q=civicrm/clearcache`) if you do not to clear the caches and rebuild the menu. This is necessary to add the CiviProxy settings page to CiviCRM.
+* you should now be able to access the CiviProxy Settings page with Administer/Administration Console (in the menu section System Settings) or with the URL `<your civicrm website>/civicrm/admin/setting/civiproxy` (if you have CleanURLS enabled or if you do not `<your civicrm website>/?q=civicrm/admin/setting/civiproxy`). 
+
+!!! note
+    If you want to you can add the CiviProxy Settings page to your Administration menu with Administer/Customize Data and Screens/Navigation Menu.
+    
+* check the [Configuring CiviProxy](configuration.md) page to see how to configure the CiviProxy extension. 
+
+## Installing the Proxy scripts your your CiviProxy server

docs/outside.md

@@ -0,0 +1,4 @@
+# What to do if an outside application wants to communicate with CiviProxy
+In most cases when an outside application (for example the public website, or maybe even a  center webservice) wants to access CiviProxy to be able to retreive data from CiviCRM or send data to CiviCRM they will want to use the API. 
+
+In that case you should provide them with the Site key and an API key they can use. They will have to use your CiviProxy URL in their REST request, and you will need to provide them with that URL.

docs/requirements.md

@@ -0,0 +1,9 @@
+# Technical Requirements
+
+There shouldn't be any requirements that any web hoster wouldn't comply with, but here they are:
+
+ 1. PHP 5.3+
+ 2. PHP PEAR (to install on Debian/Ubunto, run `apt-get install php-pear`).
+ 3. Read/write permissions on your webspace
+ 4. Reasonable amount of protection, i.e. only authorised users (you) can upload/download the files
+ 5. Ideally with it's own IP address (makes configuring the VPN easier)

mkdocs.yml

@@ -0,0 +1,23 @@
+site_name: CiviProxy Guide
+site_description: Guide using and installing CiviProxy to manage the traffic to and from your CiviCRM installation
+repo_url: https://github.com/systopia/CiviProxy
+theme: material
+pages:
+- Introduction: index.md
+- Technical Requirements: requirements.md
+- Installing CiviProxy: installation.md
+- Configuring CiviProxy: configuration.md
+- Directing an outside application to CiviProxy: outside.md
+- Future enhancements to CiviProxy: enhancements.md
+
+markdown_extensions:
+  - attr_list
+  - admonition
+  - def_list
+  - codehilite
+  - toc:(permalink=true)
+  - pymdownx.superfences
+  - pymdownx.emoji
+
+
+