DCloud Instructions
Overview
Usually, setting up DCloud will involve at least 2 installations:
- A Subversion or Mercurial repository on a host available via the Internet, along with supporting software.
- One or more client machines.
Alternately, it can be use local documents without a repository, although this defeats the cloud-like characteristics.
Dependencies
Clients:
- Korn shell
- Subversion or Mercurial (recommended)
Server/clients with web publisher installed:
- Korn shell
- Subversion or Mercurial
- Python3 (3.5 or later) with pytz, localtz, ics, & arrow
modules (installed via
pip
/pip3
). - Make
- m4 macro processor
Subversion
Setting up a Subversion Server
Before setting up a Subversion repository for your cloud, create
a new user svn
on the desired node. Make sure
svn
has access to Subversion (svn
,
svnadmin
, svnserve
), Python 3
(`python3'), Korn Shell (ksh
), and their own
$HOME/bin
(aka ~/bin
).
As that user, download and extract the DCloud tarball. Then, in resulting directory:
$ make install_svn
This will install:
- Some utilities and a
post-commit
hook in~/bin
. - A repository named
cloud
, in~/Repositories
. - DCloud website template code in
~/Sites/Cloud
.
Configure the repository (Subversion)
Next, configure repository users by editing editing 3
configuration files in Repositories/cloud/config
.
In passwd
, add something like:
christine = some_password
publisher = another_password
In authz
, add something like:
[cloud:/]
christine = rw
publisher = r
# Prevent publication of private stuff
[cloud:/Private]
publisher =
Revise svnserve.conf
with these options:
anon-access = none
password-db = passwd
authz-db = authz
realm = Name Your Cloud
Configure repository access (Subversion)
If you want to set up svnserve
to run as a service,
or have it invoked by
launchd
/systemd
/inetd
on-demand, refer to your system and Subversion documentation. These
methods do not use encryption; that requires Apache.
To use Subversion over SSL (svn+ssl), use
ssh-keygen
(or the appropriate utility on your system)
to create a public/private key pair. In ~/.ssh
,
retrieve the public key just constructed (named
id_something.pub
). Then, add or edit the
authorized_keys
file in the same directory.
Repeating that as a cookbook:
$ mkdir ~/.ssh
$ chmod 700 .ssh
$ cd .ssh
$ ssh-keygen
Follow ssh-keygen’s prompts; refer to man pages as necessary.
$ cat id_rsa.pub
ssh-rsa AAASeveralLinesOfAlphabetAndNumbersSoupHereAQ32 perette@lenora
Select and copy the key, including the ssh-rsa
and
user@host portions.
$ touch authorized_keys
$ chmod 600 authorized_keys
$ your_favorite_editor authorized_keys
Append to the file:
command="svnserve -t tunnel-user=publisher -r Repositories" [pasted key]
Save, quit, and test:
$ ssh localhost
( success ( 2 2 ( ) ( edit-pipeline svndiff1 absent-entries commit-revprops depth log-revprops partial-replay ) ) )
Use Control-D to terminate the session. If you don’t get
something that looks vaguely lisp-like, diagnose and fix. If
svnserve
is not in /bin
or
/usr/bin
, you may need to provide a full path to it in
authorized_keys
.
Checkout the repository (Subversion)
Create a working copy of the repository. After changes are submitted, this will be used to extract a copy to generate a web-accessible version of your site. If you set up Subversion with the suggested svn+ssl setup:
$ cd ~/Documents
$ svn co svn+ssh://localhost/cloud Cloud
Checked out revision 0.
Configuring Subversion Clients
To set up a client, download and extract the DCloud tarball. In the resulting directory:
$ make install_client
This will install a few utilities in ~/bin.
Follow the procedure “Configure repository access”,
above, to create a private/public key pair for this node. Add the
public key (in id_something.pub
) to the Subversion
user’s authorized_keys
on the server,
similar to before but using the user’s assigned Subversion
name:
command="svnserve -t tunnel-user=christine -r Repositories" [pasted key]
You can now:
$ ssh ssh@the_svn_server
( success ( 2 2 ( ) ( edit-pipeline svndiff1 absent-entries commit-revprops depth log-revprops partial-replay ) ) )
^D
$ cd ~/Documents
$ svn co svn+ssh://svn@the_svn_server/cloud Cloud
Checked out revision 0.
You can now put documents into your cloud. You can move
documents or directories there, then use svn add
to
add them; or you can use cloud edit -c
to add new
files. Use cloud -?
for more information on that
utility.
Mercurial
Setting up a Mercurial Server
Before setting up a Mercurial repository for your cloud, create
a new user hg
on the desired node. Make sure
hg
has access to Mercurial (hg
), Python 2
(python
), Python 3 (`python3'), Korn Shell
(ksh
), and their own $HOME/bin (aka ~/bin).
As that user, download and extract the DCloud tarball. Then, in resulting directory:
$ make install_hg
This will:
- Install some utilities and the
post-commit
hook in~/bin
. - Create a repository named
cloud
, in~/Repositories
. - Configure the
commit
hook to the repository’shgrc
configuration file. - Clone the repository in ~/Documents/Cloud
- Install DCloud website template code in
~/Sites/Cloud
.
Configure the repository (Mercurial)
The install Makefile sets up a usable single-repository,
single-user configuration. If you want multiple repositories or
multi-user access to your cloud, you will need to research adding
users to Mercurial. Look into hg-ssh
,
hg-ssh2
, and hg-ssh3
, which seem to be
variations on a theme.
If you want to improve this manual by writing up and sharing instructions on how to do make this all work, that would be lovely.
Configure repository access (Mercurial)
To enable remote access to your repository, follow the same
“Configure repository access” procedure as for
Subversion, but with the command=
string in
authorized_keys
:
command="hg -R Repositories/cloud serve --stdio" [pasted key]
It can be tested by making a clone, below.
Configuring Mercurial Clients
To set up a client, download and extract the DCloud tarball. In resulting directory:
$ make install_client
This will install a few utilities in ~/bin.
Checkout the repository (Mercurial)
Once access has been set up, configure a client by:
$ hg clone ssh://the_hg_host/ ~/Documents/Cloud
requesting all changes
adding changesets
adding manifests
adding file changes
added 4 changesets with 13 changes to 7 files
updating to branch default
7 files updated, 0 files merged, 0 files removed, 0 files unresolved
You can now put documents into your cloud. You can move
documents or directories there, then use hg add
to add
them; or you can use cloud edit -c
to add new files.
Use cloud -?
for more information on that utility.
Standalone
To install DCloud without a repository back-end, make sure your
node has Python 3 (`python3') and Korn Shell (ksh
).
Download and extract the DCloud tarball. Then, in resulting
directory:
$ make install_standalone
This will:
- Install some utilities and the
post-commit
hook in~/bin
. - Create a repository named
cloud
, in~/Repositories
. - Install DCloud website template code in
~/Sites/Cloud
.
Add to your .kshrc and/or .bashrc:
export DCLOUD_BACKEND=none
And create your (not-so) cloud document directory:
mkdir ~/Documents/Cloud
Web View Support
Set up web access
DCloud’s post-commit
hander generates a set
of simple, friendly, static web pages in
~/Sites/Cloud
. Afterwards, if it exists,
upload
is invoked with the parameter
‘cloud’. Implement this script to transfer updates to a
web server.
For example, use your hosting company’s CPanel to create
cloud.mydomain.com
. Log into your hosting account, and
add the repository user’s public key to authorized_keys (do
not include any command=...
stuff). Your
upload
script might then look something like this:
#!/bin/ksh
rsync -lptz -r -H -e "ssh -p 22" ~/Documents/Cloud/ my_server.hostingcompany.com:cloud.mydomain.com
Remember to chmod u+x ~/bin/upload
. You can test it
by invoking it on the command line.
Configure behavior
You can set pathnames, calendar categories, magic filenames, and
case of output filenames, and other settings in ~/.dcloudrc.
Available settings are listed at the top of the
dcloud-post-commit
script.
Restrict access
Configure restricted access in your hosting provider’s
CPanel. This usually results in a .htaccess
file in
the served directory, which must be copied to
~/Documents/Cloud
to prevent it being deleted by
rsync
.
Formats vary; consult your hosting provider’s
documentation for details if you need to create or customize
.htaccess
.
Test well to ensure security.
File Formats
The web publisher recognizes text documents, markdown, RTF, PDF, and various graphics formats. For some formats, you need additional utilities:
- Markdown (.md): Install multimarkdown from
http://fletcherpenney.net/multimarkdown/
If not available, Markdown is converted as plain-text. - RTF (.rtf): Install
unrtf
, available viaapt
, MacPorts, and other fine package managers. If not available, RTF files are omitted from web access. - Unidentified (no extension): the document is probed for Markdown formatting; if found, it is processed with Multimarkdown (if available). Otherwise, it’s treated as plain-text.
Customizing
Handling transient server unavailability
You may want to add a periodic cloud sync
to your
clients' crontab
.
Cleaning and archiving of completed to-do tasks
I have a nightly job that extracts completed tasks from the to-do list, and puts them in a completed-tasks list:
#!/bin/ksh
# Extract done stuff from my to-do list, and keep a log of it.
if cd ~/Documents/Cloud
then
cloud sync
tasks.py -f markdown -c -b -m "Completed $(date '+%Y-%m-%d %H:%M')" -s tasks.txt >> completed.txt
cloud sync
fi
Helpful aliases
You’ll probably want to add some aliases to ease use. Here’s a few of mine:
alias tasks='cloud edit tasks.txt'
alias queue='cloud edit queue.txt'
alias grocery='cloud edit Purchasing/grocery.txt'
alias brain='cloud edit brain.txt'
tasks.txt
is the to-do list. Tasks are put in a
markdown multi-level bulleted list:
- [ ] Incomplete task
- [x] Completed task
queue.txt
is a text-file based calendar.
Both tasks.txt
and queue.txt
are
converted to HTML and put on the index page of the cloud website.
queue.txt
is also converted to a calendar.
brain.txt
is filled with assorted notes, much of it
system-administration related but not entirely. It’s a sort
of swapfile for my brain; it’s got all the stuff I use rarely
enough that I need to look it up every time: Vim modelines, public
wifi passwords, part numbers for wood chipper blades. Mac boot
key-chords, and which services to enable after a fresh install to
make a Mac behave correctly. A list of packages to install on a
fresh LinuxMint. Dimensions for making fetish items. How to spell
the word ‘tschochke’. You get the idea.
Improving reponsiveness
The Subversion repository is configured with the post-commit hook run in the foreground, so that errors are reported to clients in real-time. Commits can be sped up by deferring web conversion, with the peril that errors may go unnoticed. Modify the invokation of dcloud-post-commit in Repositories/cloud/hooks/post-commit:
batch << EOF
~/bin/dcloud-post-commit "$1" "$2" > $HOME/Logs/post-commit.log 2>&1
EOF
Box pictures instead of checkboxes in to-do lists
Have a Samsung phone that shows a picture of a ballot going in a box instead of a box with/without a check? I consider this a defect by some literalists in Samsung’s font department.
A workaround is to add this CSS to custom.css
to
use a crossed or uncrossed circle:
UL.todolist LI:before {
content: '\25ef ';
}
UL.todolist LI.completed:before {
content: '\2bbe ';
}
However, these may pose problems on iPhones. Another choice is a checkmark and a bullet:
UL.todolist LI:before {
content: '\2219 ';
}
UL.todolist LI.completed:before {
content: '\2713 ';
color: green;
text-decoration: none;
}