What’s Within?

This post lays out how I research topics. I’ve struggled with “research” in the past and, thanks to a friend, finally figured out a methodology that actually works for me. This post goes through my methodology, how I use it with success and the details needed to recreate and/or adapt it.

Table of Contents

What / Why / How

Important Context

Over the years I never really figured out how to research a topic properly. Basically if I needed to research anything I just sort of went digging around the internet, read books, read published articles and similar but didn’t really take notes or create summaries or similar. At best I’d catalog very basic snippets that I could turn into something actually useful at a later date.

This scattershot, unstructured approach proved problematic many times over the years. Every time it bit me in the ass I went looking for information on how others performed research but I never really found a methodology that worked for me.

This approach was particularly problematic whenever I tried to look into some of my ongoing health hassles. Thankfully a friend clued me into some options for researching health hassles. With this new found knowledge I was able to do some digging and I found a way of performing research that actually works for me.

This post lays out the research method that I use. It’s a methodology that I instinctively gravitate to and has become something of a default that I reach for whenever I go looking for information on a topic. No matter the topic or information density, this has become my starting point.

As always: use what works, trash what doesn’t if you try anything put forth here.

Holy Potatoes Batman!

Before getting into the specifics, I should point out that while acting upon the information my friend gave me, I ended up using some of the very tools suggested to figure out how to research a topic.

Specifically my friend suggested a few applications that are used heavily (and adored) by academic researchers. After a quick review of the options put forth and others I found while looking at alternatives (for completeness only, I trust my friend to make solid recommendations), I made a judgement call and selected Zotero as my starting point.

Given this topic was framed as “researching health hassles” I started pulling any open access research related to my health hassles into Zotero. As I was doing this I realized I had only sorted the “catalog what looks interesting and/or useful”. Helpful for sure, but definitely not helpful when it comes to reading, distilling and working towards an understanding of a dense topic.

It was at this point I thought: “what the hell, can I use Zotero itself to figure out how to use Zotero in a way that works for my needs?”.

Yes, yes you can use Zotero to figure out how to research a topic and to help develop an actual, usable research methodology.

I literally used Zotero to figure out how to use Zotero successfully. It looked like I was working with an Ouroboros but in reality I was using a compiler to compile itself.

A Distillation Win

Another secondary item that greatly facilitated my success with developing a research methodology was the fact I was in the midst of migrating my 2nd Brain (digital common book) from org-mode to Obsidian.

I had run into problems with org-mode and was seeing phenomenal success and improvements with Obsidian, so much so that I instinctively thought “I bet I could use Obsidian to distill research into a more usable format for the long-term and/or for publishing”.

Having Obsidian as a tool in the box proved to be critical.

As soon as I looked into ways of tying Zotero and Obsidian together I immediately found a few articles that laid out exactly what I needed and desired. These articles also clued me into the final, critical, component I needed for developing my personal methodology.

Notes, Annotations and Attachments. Oh my!

The articles I found that outlined how to tie Zotero and Obsidian together focused heavily on notes, annotations (like highlights) and attachments. I found this confounding at first but after a few reads and poking around, I realized the key to research is finding an approach to note taking and annotations that works for an individual.

In my case it was realizing that I could highlight inline, add notes related to these highlights and effectively create a “catalog of interesting” for each item within Zotero. I already did this in a basic form but never in a way that’d be considered useful.

I decided to put this to the test and very quickly generated a large catalog of interesting related to using Zotero for research. I was finally seeing the beginnings of my methodology take shape.

This was the final, critical component I needed to achieve success in developing my personal research methodology.

Organizational Hiccups

Despite my initial success, I quickly run into multiple problems with organization.

I have multiple areas of interest with multiple, active areas of research these days. This necessitated developing an organizational scheme that allows me to focus on individual topics. Especially given I struggle when presented with a list of generally unrelated items.

I require “zones of focus” to be successful. I need to keep related items “in one place” and have “multiple places for information” so I can better focus and avoid the overwhelm that happens when things aren’t at least loosely organized by topic.

I ended up creating a set of Zotero collections, one per topic, that I use to stay focused within Zotero. I also have a __Templates collection and a _Completed collection. I use __Templates to ensure any notes that I always attach to a Zotero item are consistent, particularly my “Main Take Aways” note. I use _Completed to hold any collections that I feel are fully researched and appropriately distilled within Obsidian.

I also discovered that my notes within each item were a scattered mess. I needed a way to layout the notes in a form that was useful for me with a good overall order. Thankfully I’m a huge organizer and notebook nerd. I had read about Johnny Decimal and similar systems in the past but never put them into practice as I tend to organize my information in a less rigid form. However, within Zotero I knew the more rigid nature of Johnny Decimal (and similar) would likely be beneficial.

I re-read the entirety of the Johnny Decimal website and felt I could use the information to build a similar system for my Zotero needs. After some fiddling and tuning I was able to put together a layout that tamed the chaos of my notes and attachments for items stored within Zotero. It allowed me to have a meaningful order of items and the items are grouped in a meaningful way.

Once I sorted these two organizational needs I had everything I needed.

Tying Zotero and Obsidian Together For Distillation

Once I had my catalog of interesting created and organized, for each item within Zotero, I finally got around to tying Zotero and Obsidian together.

Within Obsidian I use a plugin to draw down all notes and annotations with a list of attachments into a single note for a given Zotero item inside my Research folder. Basically I use the Obsidian plugin to draw down the catalog of interesting for a given Zotero item as an individual note per item. I can then look at all the Obsidian notes for a given research topic and see all the information I felt pertinent for distillation.

Once I have everything drawn out of Zotero I can then start the process of distilling everything into a single note within Obsidian for long-term use, publishing, etc. I go through everything, start laying out the information in a cohesive form that flows well and I ultimately turn snippets and pieces of information into something a bit richer and more fully explained.

Exactly what I was hoping to accomplish.

This Works

As soon as I had a usable, general workflow developed; I almost immediately started to instinctively use Zotero to manage my research. I setup collections for each health hassle and started to add open access research articles, I created a _To Sort collection for items that didn’t necessarily fit into an existing collection but were interesting, I created collections for my personal projects (mostly tech) and more. I eventually setup a Misc collection for research articles and other, dense information I expect to annotate but don’t necessarily fit into my existing collections.

It worked so well that when I decided to setup a Cat TV, I did all of my research and early digging around wholly within Zotero. Once I had all the major information cataloged and annotated, I drew down the information into Obsidian. From there I was able to distill everything into a how-to and technical reference that will allow me to re-create this Cat TV setup in the future.

Using Zotero with Obsidian for the Cat TV project also showed me just how much information I review while working on a project. It also showed me just how heavily I reference back to my initial research as well as anything noteworthy within my research. I also noticed the amount of time I spent getting this project off the ground and completed was less than I would normally put forth. I gained a lot of efficiency and reduced time commitment thanks to my research methodology. Funny enough, I wasn’t trying to improve my efficiency but I’ll take the win.

I had finally figured out how to research in a robust and meaningful way.

Fun Fact

In a truly “yep, that’s Kemo’s brand of B.S.” moment: this blog post is backed by an entire Zotero collection, a set of distilled Zotero Obsidian notes and a Quick Reference I put together based on my research.

Using Zotero and Obsidian to figure out how to research ended up allowing me to figure out how to research in a way that works for my brain. There’s something slightly mind bendy about using tools you don’t know how to use to figure out how to use the tools themselves…

The High Level Overview

To research topics and distill them I use Zotero combined with Obsidian. This may sound simple but, as always, the details are important.

I use the following process to research a topic:

  1. Use the internet to look into a topic
    • General purpose search engines
    • Science paper search engines like the NIH website
    • Specialized forums
    • Wikipedia
  2. Import / send information into Zotero
  3. Read collected information
    1. Create “main take away note”
    2. Annotate / highlight as I read information
    3. Create additional notes and attachments as needed
  4. Import information into Obsidian
  5. Distill and write up everything as a dedicated Obsidian note, blog post, etc
  6. Archive Zotero collection
  7. Archive imported data in Obsidian

The Specifics

The rest of this post is essentially a quick reference for my Zotero and Obsidian setup. The overall approach is nice to know but using such things with success requires knowing some amount of finer detail.

Given this is pretty universal setup, I’ve included the key information needed to recreate and/or adapt my methodology below.

Obtaining Software & Plugins?

At the end of this post is a Links section with links to everything discussed in this post. If you’re looking for something specific I’ve mentioned, scroll down to the links section.

A Quick Note About PDF Files

Simply put: PDF files are a pain in the ass to read and annotate. Especially research articles formatted in ways that clearly tell me a lot of “journals” have no idea how the digital world works.

I have no solution for this problem and the best advice I can give: fuck around with the Zotero document viewer and see if there is a way to use it in a manner that won’t make you rage out like Mr. Hyde. It can be done but takes an asinine amount of time and fiddling to figure out what will work for individual need.

If you work for a science journal: please find your head from ass and start publishing single column, consistently formatted PDF files with embedded text that can be properly highlighted.

A Quick Note About Mobile Use

Zotero’s online reader does work on mobile. It’s not the best but it’s sufficient for light use on mobile devices. It’s far more usable on devices the size of a small tablet and larger but can be used on a phone in a pinch.

There are mobile apps but I cannot speak to their utility as I use Android and the Zotero Android app is currently behind a closed beta while they develop it. Hopefully in time the Zotero Android app will become generally available and be a bit more robust than their web reader.

The enterprising may discover there is a KOReader plugin that can sync PDF annotations with Zotero. KOReader has surprisingly robust PDF support and is worth looking into if you plan to use a small tablet or phone for your research. Especially when combined with the Zotero plugin. It’s not perfect but is a great option to investigate for mobile use.

Annoyingly, not 5 minutes after completing the first draft of this post the Zotero project announced official support for importing KOReader and Calibre epub annotations. I haven’t looked into this yet but given what I’ve seen, I expect this to be a great option for mobile use if not using the native Zotero app on mobile. KOReader and Calibre annotation support should also allow e-reader devices to be used for annotations. I’ve included a link to the very specific, code level details below.

Zotero Workflow Details

This is the generalized workflow I use within Zotero for research.

  • Add item
    • New Item -> ...
    • Zotero browser connector for adding web content
  • Attach full texts / snapshots / supporting files as appropriate
  • Tag main item with kmn-to-research to flag it has not been read / annotated / etc
  • Tag main item with standard item tags as appropriate
  • Create 10.010 - Main Take Away note from template
  • Read, annotate, take notes, etc
  • Fill out 10.010- Main Take Away note
  • Ensure all sub items are organized / named according to modified Johnny Decimal layout
  • Remove kmn-to-research tag when finished with main item
  • Move collection to _Complete when research into the topic is complete and each item’s catalog of interesting has been imported into Obsidian

Zotero Collections

The below is a small snapshot of the collections I have defined within Zotero. Each collection represents a single area of research (topic). Note: some items may apply to multiple areas of research. In this circumstance I’ll add the item to multiple collections as appropriate.

  • __Templates
  • _Complete
    • _How To Research
    • Cat TV
  • Gastritis
  • Raynaud's

Individual Items Within Zotero

The below is a small snapshot of what notes, attachments and similar I have linked to individual items within Zotero.

It should be noted that highlights and annotations are stored inline with the main Zotero items and do not show as sub-elements in the list. However, if you click on an item in the Zotero list you’ll see an “annotations” section alongside the item properties with all of your annotations listed.

  • Johnny Decimal
    • 10.010 Main Take Away
    • 90.010 Whole Site As epub
    • 90.011 Homepage Snapshot
  • KOReader - User Guide
    • 10.010 Main Take Away
    • 90.010 Snapshot
    • 90.011 PDF
  • ReadEra
    • 10.010 Main Take Away
    • 40.010 Play Store
    • 90.010 Snapshot
  • Using Zotero Notes
    • 10.010 Main Take Away
    • 90.010 Video
    • 90.011 Subtitles (en / vtt)
  • yt-dlp/yt-dlp
    • 10.010 Main Take Away
    • 80.010 readme.txt
    • 80.015 build.sh
    • 80.020 Dockerfile
    • 80.025 yt-dlp.sh
    • 90.010 Snapshot

Zotero Tags

These are the main tags I use within Zotero to help me manage my research. I’ve included additional detail for each tag to ensure the purpose of each tag is clear.

  • _template
    • Items, like notes, that I want to be consistent across all Zotero items
  • kmn-collection_or_topic
    • “collection_or_topic” is the collection name itself
    • This is here to help with searches across multiple collections
    • This helps me know if an item is related to multiple collections
  • kmn-obsidian-vault-name
    • “name” is the obsidian vault where the item will be distilled
    • This helps me track if an item will be added to my “2nd Brain” or “Medical” Obsidian vaults
  • kmn-take-away
    • So I can find my main take aways for items using search
  • kmn-to-research
    • So I can find items that I’ve not finished reading and annotating

Zotero Color Mappings

I use colors to help signal intent and facilitate scanning through information looking for specific types of items. This is a list of all item colors available within Zotero as of this writing. Colors without detail are currently unused and are included for completeness.

  • Yellow
    • Warning
    • Careful
    • System level item
  • Red
  • Green
    • General
    • “1st pass” highlights
  • Blue
  • Purple
  • Magenta
  • Orange
    • Important, additional info in item comment
  • Gray

Zotero Naming of Notes, Attachments, etc

This is the modified form of Johnny Decimal that I use for naming notes, attachments, etc that are associated with a given item within Zotero.

Each of the below categories should have template notes with appropriate high level tags present.

Template file naming convention: _XX.NNN - [Category Name]

  • 10.NNN - Take Aways / Notes
    • Start at 10.011 as lowest value
    • 10.010 - Main Take Away Note
      • This should be a rough summary at most
      • If content is ‘small enough’ can be ‘more’
      • Is templated
    • Take away notes are templated
  • 20.NNN - Annotations
  • 30.NNN - Quotes
  • 40.NNN - Other Info
  • 50.NNN
  • 60.NNN
  • 70.NNN - Screen shots
  • 80.NNN - Source Code files / snippets / etc
  • 90.NNN - Snapshots / Source Material
    • Can be pdf / epub / video / page snapshot / etc
    • 90.010 - Main ‘snapshot’, PDF, ePub, etc
      • Do NOT use if multiple ‘mains"
    • Video Subtitles…
      • Are the next ID(s) following the video they are for
      • Encode the language and format as part of the ’name’
      • 90.015 Video
      • 90.016 Subtitles (eng / vtt)
      • 90.017 Subtitles (ger / pgs)

Obsidian Specifics

I won’t put forth much in the way of Obsidian specifics. I use Obsidian as a form of Common Book and there is a folder called Research where everything related to Zotero lives.

Distilled content backed by Zotero ultimately gets filed in the appropriate “location” within my Obsidian vault(s). My specific Obsidian vault layouts are nuanced, dense and well outside the scope of this post.

However, anything I have setup related to Zotero is laid out below. There isn’t much to setup within Obsidian and the below will let you get under way with using Obsidian to distill information pulled from Zotero.

Obsidian Import Configs

This is how I setup the Obsidian plugin that imports data from Zotero.

Note: I have one import format defined per Zotero collection. The example below is just one of many I have configured.

  • Import format
    • Name: How To Research
    • Output Path: Research/How To Research/{{ title | replace(r/[^A-Za-z0-9._-]/g, "_") }}.md
    • Image Output Path: Research/How To Research/_attachments/{{ title | replace(r/[^A-Za-z0-9._-]/g, "_") }}/
    • Image Base Name: image
    • Template File: Research/_Templates/KemoNine.md
    • Bibliography Style: Chicago Manual of Style 17th edition (author-date)

My Zotero Obsidian Import Template

This file is located at Research/_Templates/KemoNine.md and used for each Obsidian Zotero import configuration.

---
tags: {% if allTags %}{{allTags}}{% endif %}
citekey: {{citekey}}
---

> [!Cite]
> {{bibliography}}

>[!Synth]
>**Contribution**:: 
>
>**Related**:: 
{% for relation in relations %}[{{ relation.title }}]({{relation.title.replace(r/[^A-Za-z0-9._-]/g, "_")}}){% if not loop.last %}, {% endif%}{% endfor %}
>

>[!md]
{% for type, creators in creators | groupby("creatorType") -%}
{%- for creator in creators -%}
> **{{"First" if loop.first}}{{type | capitalize}}**::
{%- if creator.name %} {{creator.name}}  
{%- else %} {{creator.lastName}}, {{creator.firstName}}  
{%- endif %}  
{% endfor %}~ 
{%- endfor %}    
> **Title**:: {{title}}  
> **Year**:: {{date | format("YYYY")}}   
> **Citekey**:: {{citekey}} {%- if itemType %}  
> **itemType**:: {{itemType}}{%- endif %}{%- if itemType == "journalArticle" %}  
> **Journal**:: *{{publicationTitle}}* {%- endif %}{%- if volume %}  
> **Volume**:: {{volume}} {%- endif %}{%- if issue %}  
> **Issue**:: {{issue}} {%- endif %}{%- if itemType == "bookSection" %}  
> **Book**:: {{publicationTitle}} {%- endif %}{%- if publisher %}  
> **Publisher**:: {{publisher}} {%- endif %}{%- if place %}  
> **Location**:: {{place}} {%- endif %}{%- if pages %}   
> **Pages**:: {{pages}} {%- endif %}{%- if DOI %}  
> **DOI**:: {{DOI}} {%- endif %}{%- if ISBN %}  
> **ISBN**:: {{ISBN}} {%- endif %}    

> [!LINK] 
> {%- for attachment in attachments %}
>  - [{{attachment.title}}](file://{{attachment.path | replace(" ", "%20")}})  {%- endfor -%}.

> [!Abstract]
> {%- if abstractNote %}
> {{abstractNote}}
> {%- endif -%}.
> 
# Notes
> {%- if markdownNotes %}
>{{markdownNotes}}{%- endif -%}.


# Annotations
{%- macro calloutHeader(type, color) -%}  
{%- if type == "highlight" -%}  
<mark style="background-color: {{color}}">Quote</mark>  
{%- endif -%}

{%- if type == "text" -%}  
Note  
{%- endif -%}  
{%- endmacro -%}

{% persist "annotations" %}
{% set newAnnotations = annotations | filterby("date", "dateafter", lastImportDate) %}
{% if newAnnotations.length > 0 %}

### Imported: {{importDate | format("YYYY-MM-DD h:mm a")}}


{% for a in newAnnotations %}
{{calloutHeader(a.type, a.color)}}
> {{a.annotatedText}}
{% endfor %}
{% endif %}
{% endpersist %}
DailyLife  Tech  Opinion  Writing 

See also