Skip to contents

myTideR: Gotta Tiddl' Em All!

myTideR.Rmd

Introduction

A tide is the in-flow of tiddlers generated by an external script, often scheduled to be ran at specific times. Data flow in and out of your wiki. The aim is to help you build an overview of the day that has passed, in parallell with you tiddlywiki note taking; what happened in the news that day? What was the weather like? What web sites did you enjoy? Did you receive important emails? What files did you work with?

Demonstration of an early version: https://www.youtube.com/watch?v=kBH73OoaL9o

First a warning: This is mostly a proof-of-concept and should not be trusted with valuable data. Use on your own risk and keep backups!

To get started, you need to initiate the tiddlywiki server.

library(myTideR)
init("../my_tide") # path to your tide folder

This will create a tiddlywiki server at that location,1 relative to the R working directory. To start the server from R:

p <- start_server(tidepath="../my_tide")

The path can be an absolute path or relative to the R working directory. Within this folder, the package will create the following directories:

  • file-archive. Files will be moved here from the inbox
  • inbox. This is the folder you will copy your stuff to
  • outbox. Currently not used, but the idea is that your scripts can put things here
  • web-archive. Archived web sites will be saved here
  • .wiki. This is where your wiki is living

The script will also add certain tiddlers to your tiddlywiki:

  • The $:/tide/ tiddlers include custom viewtemplates, scripts and settings.
  • $:/config/Server/ExternalFilters/[prefix[$:/tide]]. To get access to the tide tiddlers.
  • $:/config/SyncSystemTiddlersFromServer. To get access to the tide tiddlers.

Note the the tides themselves will be saved with prefix $:/tides/ (not $:/tide/) to separate them from those controlling the scripts.

By default, the tide scripts are tagged with DoNotRun. You have to remove that tag to activate the scripts (or use run_tides(force_run = TRUE)).

To run your tide scripts, in R:

run_tides()

Using MWS

The MultiWikiServer Plugin is a plugin currently under development that introduces recipes and bags, and stores the tiddlywiki to a SQLite database instead of flat files. There are several benefits, including performance. To use the MWS plugin, you need to download the git repository for tiddlywiki and initiate your server as (replace "../TiddlyWiki5") with the relative path to the downloaded repository:

library(myTideR)
init("../my_tide", mws = TRUE, tiddlywiki = "../TiddlyWiki5")

You can later retrieve whether or not your tides lives in an MWS wiki by checking the tiddler called $:/tide/settings/mws:

mws <- tides[title == "$:/tide/settings/mws", text] == "yes"

Included tides

The RSS tide

The RSS channels are tiddlers with prefix $:/tide/settings/RSS/. For example,

title: $:/tide/settings/RSS/NRK Innenriks
rss_publisher: NRK Innenriks
rss_url: https://www.nrk.no/norge/toppsaker.rss
type: text/vnd.tiddlywiki
text: NRK Innenriks

Only rss_publisher and rss_url are used by the script. When ran, your tide tiddlers will have titles like $:/tides/RSS/<uuid>.

Please note that the script has only been tested with the RSS from NRK, and some adaption may be needed if you add other channels.

The File tide

The file tide will archive the files you have added to your inbox folder and create corresponding tiddlers. The files will be moved to file-archive/<file creation date>. The tiddlers will contain file metadata, as well as links to the files themselves.

There should be some protection against colliding file names and duplicates (hash), but there’s no guarantee against data loss.

Note: Chrome won’t let you open local files by default this way, you need to install an extension to open the external files from the browser.

The Hook tide

The hook tide allows you to “do things” in-text using hooks. For example,

This is some tiddler content. You can have a TODO::todo element:: or perhaps a SECRET::little secret::. Or perhaps a
SUMMARY::
This
is a
multiline
summary
::

Here, we will call TODO, SECRET and SUMMARY the hooks. They are identified by the following ::.

The default behavior is that the content will be moved to a separate tiddler, titled something like $:/tides/hook/HOOK/<uuid> (where HOOK in the example above will be SECRET, TODO or SUMMARY), and transcluded in the original tiddler as {{$:/tides/hook/HOOK/<uuid>||$:/tides/hook/HOOK}} (i.e., $:/tides/hook/HOOK is acting as template).

But feel free to implement more advanced hook actions. For example, it may be useful to have a hook for DOI::<doi>:: that fetches the doi record, creates a tiddler for it, and make an appropriate link.

The Web tide

The web tide will archive web pages listed in the reference fields of your tiddlywiki. Using single-file, the web pages are downloaded to the web-archive folder, and the reference is updated to point to a tiddler instead of the original link.

Note: You need to install the single-file cli on your system!

The Mail tide

The mail tide is quite similar to the file tide; drag-and-drop mails from outlook (.msg) to the inbox folder, and the tide will create a tiddler for the mail itself and archive the attachments (run the file tide). This tide requires the https://github.com/hrbrmstr/msgxtractr.

Please note: The tide_order field ensures that the mail tide runs before the file tide.

Ideas for new tides

The Calendar tide (not implemented yet)

Import events from your calendars and have them listed at the corresponding dates.

The YouTube history tide (not implemented yet)

Following a Google Takeout, import your youtube history to see what videoes you used to enjoy in the past.

The Spotify tide (not implemented yet)

Import your Spotify history to remember what songs you used to listen to.

The Google Maps tide (not implemented yet)

Following a Google Takeout, import your google location history to see where you spent your days.

The LogSeq/Obsidian tide (not implemented yet)

TiddlyWiki is not the perfect interface for everything. However, it is perfect to get an overview of all your data. This tide imports your LogSeq/obsidian files (intended as read-only).

The Zotero tide

Makes an entry in tiddlywiki for all entries in your Zotero database.

Making custom scripts

In general, the tide scripts should look like this:

```r
do_some_stuff <- function(tiddlers, tides) {
  ...
}

do_some_stuff(tiddlers, tides)
```

where tiddlers and tides are datatables that are provided by the myTideR packages (note that you can keep the backticks!). That is, all tiddlers and tides are loaded once when you run run_tides().

Note: You do need to include do_some_stuff(tiddlers, tides) at the end of your script to actually run the code

Some more notes:

  • Be careful about changing pre-existing tiddlers with your tides. Since the tiddlers are only fetched once for each run_tides(), there is a risk of the local copy getting out-of-sync from the server and that you rewrite the same tiddler multiple times.
  • As an example, the file inbox folder path can be fetched read as full_file_path <- normalizePath(file.path(tides[title == "$:/tide/settings/path", text], "inbox"), mustWork = FALSE)

Using MWS

If you want to use the MWS plugin, your scripts may benefit from defining a tide_recipe field that defines which recipe to write the resulting tiddlers too. For example, the RSS tide put its tiddlers in a specific recipe:

mws <- tides[title == "$:/tide/settings/mws", text] == "yes"
recipe <- ifelse(mws, tides[title == "$:/tide/scripts/RSS", tide_recipe], "default")

Backlog/TODO

  • Make option to specify how often a tide should run
    • Requires both a read and a write to the tide scripts file
  • Write tide log files to tiddlywiki
  • Make import for gmail files as well