This tutorial will walk you through how to use Scroll for data analysis and visualization, from basic concepts to advanced techniques.

Let's dive in!

Scroll comes with several sample datasets. Let's start with the famous iris dataset:

You can also load datasets from Vega's collection:

Let's explore some basic operations on the iris dataset:

This gives us summary statistics for each column.

Let's look at filtering:

Let's start with a simple scatterplot of the iris data:

Let's look at some time series data:

Let's create a bar chart showing precipitation:

Let's look at some more complex transformations:

Let's add some computed columns:

Let's create a heatmap of annual precipitation values:

You can create multiple visualizations:

This tutorial covered the basics of data science with Scroll. Some key takeaways:

• Scroll makes it easy to load and manipulate data
• Visualizations are simple to create and customize
• Complex transformations can be done with simple commands
• Everything is readable and version-controllable

Scroll is a language for those who believe that what they cannot build with as few pieces as possible they do not understand.

Scroll is a symbolic language.

A language you write with symbols.

A symbol is a mark repeated to correlate to a pattern in nature.

A mark is a discoloration of a surface.

A symbolic language is a collection of symbols and symbol sequences that can be used to trigger the same electrical phenomena in the brain of an observer that seeing that pattern occur in nature would trigger.

By writing documents.

It is a collection of marks on a surface.

It is the syntax that Scroll uses.

It is a set of rules for how to break up a document into symbols.

It is a file.

It is a digital representation of a document as a series of bits.

A bit is a persistent location in space that has 2 states: a 1 or 0; a true or false; an on or off; a black or white; a hot or cold; a high or low.

A character is a 2D mark that maps to a unique fixed length bit sequence.

A map of bit sequences to characters.

UTF8. For now.

A word is a list of characters delimited by a word delimiter sequence.

An atom is another word for word in the Scroll language.

A line is a sequence of atoms in the Scroll language delimited by a newline sequence.

A particle in Scroll refers to an object that contains both atoms and a collection of particles referred to as the particle's subparticles.

Yes.

A Parser is a particle that consumes certain other particles to perform transformations.

A Parser can perform a variety of transformations but the most common are to:

• compile one particle in one language to another language.
• format a particle into a standardized format
• check a particle for errors
• evalute the instructions in a particle and return the results
• execute commands contained in the particle

The syntax of Scrolls is Particle Syntax, or just Particles for short.

Partilces tells you how to divide a binary sequence into parts.

• A sequence of binary bits is converted into characters using UTF8
• Atoms are sequences of characters separated by a single space.
• Particles are a sequence of atoms separated by a newline.
• Subparticles are indicated by a single indented space.

• Just indent a particle by 1 space more than its parent particle.
  • For example, because I started this line with one space more than the line above, this line is a subparticle of that line.
    • And then this line, starting with 2 spaces, is a subparticle of the line above it.
  • But this line is a subparticle of the line above that starts with "Just"

Yes. But generally when we talk about PPS we are talking about the form designed for binary and computers.

Parsers is a symbolic language where one defines Particles that can consume other Particles. It is a language for making other languages.

Scroll is a language made out of Parsers.

The purpose of the Parsers Language is to make Scroll.

The purpose of Scroll is to help humans communicate with humans and machines.

By writing Scroll documents (aka Scroll Programs) to create things.

Knowledge bases, blogs, websites, web pages, charts, animations, data science reports, ebooks, web forms, maps, that sort of thing.

Scroll helps you better understand what you are writing about and anything HTML can make Scroll can make in fewer symbols.

Scroll is a language for those who believe that what they cannot build with as few pieces as possible they do not understand.

ScrollHub is a super server for creating and publishing websites instantly using Scroll.

Cue refers to the first atom in a particle.

When communicating you want to communicate the most important information first, so the cue is often the most important information.

Most parsers use the cue to determine whether they should consume a particle or not.

Yes.

Regular expressions.

Currently no.

Parsers does not have enough functionality yet to be able to write parsers that provide everything Scroll needs.

The primary implementation of Parsers currently is half Parsers, half Javascript.

The Parsers program is parsed and compiled by a Javascript implementation of Parsers which generates a Parser and compiler for Scroll programs.

If you've mastered Javascript then using Parsers should be easy once you get the hang of it.

Yes.

No.

Yes. Parsers has atom types.

Scroll and Parsers implement many of the best ideas from Lisp, and leave out a lot of unnecessary things (like the parens).

Both.

October 1, 2024 — If you want to make web forms the inefficient way, there are many other places to read how to do that.

If you want to learn how geniuses are doing web forms, read on.

Building forms for one-field-at-a-time data entry is inefficient.

People can't copy/paste your forms. E-receipts are a pain. Auto-form fill works poorly.

It's a huge waste of your users' time that retards them and your business.

And it will always be this way.

Everything becomes radically simpler. People can do things like copy/paste entire applications; email them; version control them; collaborate on them; all effortlessly.

And guess what? You can still also compile to a one-field-at-a-time HTML form and allow users to swap back and forth! It's pure win, no loss.

This now ships in Scroll. Here is a test form. You can also see it in use in many of our products.

Because this is new technology, it is not for those that need spoon feeding. Believe it or not, making web forms also used to be complicated.

(If you need this urgently, don't have time to figure it out yourself, and need to hire professional consulting, feel free to get in touch.)

I'm tired of reading all the whiners commenting about how our CSS doesn't look so pretty yet, or the tiny little feature you need that isn't supported.

I don't care much to hear from whining losers.

I want to hear from winners. From builders. From those who strive for genius.

Beautiful designs, autoincrement, auto timestamp, joins, decision trees, all that stuff is coming. The world is gonna love this.

If you are a winner, a builder, a genius, join us on GitHub or Reddit.

September 2, 2024 — Today on HackerNews Kevin Damm had a great idea: put a <link> tag on blogs to point to the git Source Code Repository, much as blogs today have <link> tags pointing to their RSS feeds.

I've added this feature to Scroll and it's live now. If you View Source of this page, you'll see:

Client applications can start looking for and taking advantage of these tags.

You get a copy of the entire works of your favorite authors that is blazing fast, works offline, is searchable, transformable, censorship resistant, auditable, more trustworthy, ad free, tracker free, et cetera.

There is no comparison to the old ways of doing things.

vladimyr on bluesky dug up a couple of cool examples of prior art.

I've recorded a short video (1 minute version; 10 minute version) about our recent work which has begun to eat the software world.

Whether you are an experienced, aspiring, or casual programmer, yes.

If you are a writer, yes.

If you are a thinker, yes.

If you are a builder, yes.

If you choose to always remain illiterate to keep your brain different than the crowd, then no. (I applaud your brave creative choice and as a scientist I'm curious to see what happens!)

One liner: a syntax-free syntax for splitting files into particles (separated by line breaks) which can have atoms (separated by spaces) and subparticles (indented lines).

Vibe

Playground

LeetSheet

One liner: Scroll is a language for scientists of all ages where you write and combine particles (written in Particles) to evolve and publish your most intelligence ideas to HTML, PDFs, CSVs, JSON files, movie files, audio files, slideshows, charts, books, et cetera.

Vibe

Playground

LeetSheet

One liner: Parsers is a programming language where programmers write parsers (written in Particles) which consume particles (written in Particles) and also contain logic for translating those particles into actions on computing machines.

Vibe

Playground

LeetSheet

ScrollHub

Subreddit

August 9, 2024 — Tables, aka spreadsheets, are arguably the most important visual thought tool.

But no one has designed the perfect textual language for manipulating them.

Until now.

Tables evolved over the past decade primarily by:

• just doing what Hadley Wickham did in dplyr
• using Mike Bostock's d3 under the hood
• reading all of Jeffrey Heer's papers
• thinking about what would be the best tool for Max Roser and Hannah Ritchie.

August 2, 2024 — I like collections of interesting numbers, such as BioNumbers. Animated counters provide a useful perspective when pondering large numbers. I wanted to make it as easy as possible for anyone to create these counters.

So, today I added a new parser to Scroll: counter.

July 28, 2024 — Do you love those GitHub Activity Charts and want to make your own but don't know what they are called or what library to use and your work is busy and you have kids and sorry I'll call you back someone is crying?

Introducing Scroll's newest parser: heatrix!

heatrix let's you craft custom heatmap visualizations with the fewest keystrokes possible.

Heat Map + Matrix = Heatrix

heatrix is a microlang that let's you customize your visualizations using "directives":

If you like heatrix and want to see more stuff like this, but a folder on the World Wide Scroll beta and good things will come your way:

July 16, 2024 — Imagine a single plain text file named contacts.scroll where you stored the contact info for your family, friends, and colleagues, and you could track changes with git, compile it to a beautiful PDF or HTML page with search and sort, and it would also compile to CSV and/or JSON for import into iPhone, Android, Gmail, Outlook, et cetera?

Also imagine that this tool is free, open source, and public domain.

Introducing Contacts, a microlanguage (currently 23 lines of code, including comments) that makes it as concise as possible to add, prune, display, print and export your contacts.

There's almost nothing to learn (the code is self-explanatory).

In fact, it's almost guaranteed that using Contacts you will need to write less (we've taken every unnecessary character out).

1. Create a file named something like contacts.scroll with this content:

1. Run scroll build

Done! You should now see a contacts.html file for display and printing as well as TSV, CSV, and JSON files.

What do you think? Anything that should be added/removed/improved?

Woohoo!

Your blog is on top of Reddit!

OH NO.

There's a typo in your url. 🤦

What do you do?

You try to calm yourself down.

Almost no one looks at the url and the content of the article is what's important and the traffic is coming

But then you think all I can see is that i before e after c!

But then you think is that really a rule and maybe that's how it's spelled in London?

You can't help yourself.

It's an affront to your craftsmanship to not correct the bad filename.

You know you can so easily rename the file but then existing links will break and so you'll have to add redirects but that means more files to maintain or god forbid that would require adding a server and you just love your static blog and it's almost 2pm and you have to leave and pick up the kids from school what do you do!?!!

Or do you just break the links and going forward everyone will have the proper URL but everyone who clicks the old links will see an unhelpful 404 and you reflect on how great this thing that Tim Berners-Lee gave to the world is and how could you litter his creation with unhelpful 404s?

If the story above resonates with you, the bad news is you may suffer from Broken Link Phobia. Luckily, there is now a cure.

Scroll now comes with a way to generate a very helpful 404 page so your visitors will get redirected to the right place even if the URL they clicked is off by a few characters.

The best part? It works entirely client side on static sites.

Step 1 Create a sitemap:

Step 2 Create a 404 page:

That's it! The helpfulNotFound parser in Scroll will add Javascript to the 404 page that fetches all the urls found in the sitemap and shows the user the closest match. As a bonus, you also get a sitemap.txt file great for Google SEO and other uses.

See it in action here by clicking on this intentionally broken link:

I've been using Helpful 404s on all my sites for the past couple of months and it's been delightful.

I no longer spend any time worrying about making improvements to URLS.

I just do it and I know that users following old links will still have a good experience.

If you like stuff like Helpful 404's, there's a lot more like that in Scroll, which is public domain and open source.

Try it today, and if you like what we are doing please consider bringing your site to the World Wide Scroll!

ScrollSets are very useful once you get going. But starting from scratch could be a bit tedious.

Not anymore!

Just drop your CSV or TSV into the textarea below to generate a ScrollSet.

(P.S. using LLMs with ScrollSets is another great way to get started)

July 1, 2024 — Regardless of what languages you write your programs in, you probably maintain files like changeLog.txt or releaseNotes.txt.

Change logs are very helpful for:

• informing your users of new features and bug fixes
• providing detailed technical information to your development team (and open source contributors)
• periodical data analysis for strategic project planning - how fast are we shipping new features? fixing bugs? breaking things?

What if there was 1 language that let you do all 3 things at once?

Introducing Changes, a microlang (only 70 lines of code, including comments) that makes it as concise as possible to write, read and analyze change logs.

Changes generates a pretty HTML file for your end users; allows for including unlimited detailed technical information for your developers; and generates summary statistics as TSV, CSV, and JSON for your project managers.

There's almost nothing to learn (the code is self explanatory).

In fact, it's almost guaranteed that using Changes you will need to write less (we've taken every unnecessary character out).

We use Changes for the Scroll Release Notes.

Be sure to check out the source code that generates that HTML.

1. Install Scroll

1. Create a release notes file:

1. Run scroll build

Done! You should now see a releaseNotes.html file as well as a TSV, CSV, and JSON file.

What do you think? Anything that should be added/removed/improved?

June 24, 2024 — I engineered on many innovative data science tools, including Grapher and Ohayo^[1].

So I was excited to see Observable launch something new: Plot.

Plot is already great on its own. It handles many of the hard parts of data vis, and it's open source, so we can help them fix the parts that are still a little rough.

But I found a way to take Plot to the next level: I combined it with Scroll.

The Plot/Scroll integration is early, but I can already tell it will evolve into a very useful data science tool.

You can now make scatterplots with 2 words.

Obviously you want to add a few more words, but I don't think I'm exaggerating when I say that this is the simplest way to generate a scatterplot now and forever will be (until the day when we have mind-reading machines that can generate a scatterplot with zero words).

But enough talk, let's see some code. A tutorial is below.

To follow along, create a new website in 1 second using ScrollHub or install Scroll locally with npm install -g scroll-cli.

That's it (for now)!

If you like where this is headed, give us a star on GitHub.

[1] The difference between programming and engineering? It takes programming to make a program that works. It takes engineering to make a program that barely works.

June 23, 2024 — Regardless if you specialize in React, Rails, Django, Next, Java, C#, or ObjectiveC, you probably use templates to start new projects.

Templates generate a handful of files like readme.md, .gitignore, and main.

They also initialize a handful of directories like src/ and tests/.

What if it was even easier to make, edit and use these templates?

Introducing Stamp, a microlang (only 60 lines of code, including comments) that makes it as concise as possible to write, edit, share and expand project templates.

1. Install Scroll

1. Create myFirstStamp.scroll

1. Run scroll build

Done!

Stamp was jointly created by me and Guillaume Papin. Other tools (listed below) do similar things, but with some slight differences.

by Breck Yunits

HTML | TXT

April 29, 2024 — Scroll is a new language for building HTML and CSV files that powers blogs, websites and knowledge bases.

I often need plain text outputs in addition to HTML files.

I used to make those plain text versions copying and pasting text by hand.

Now Scroll has plain text output built-in.

Add a single line: buildTxt

scroll build will create example.txt. Done!

You may want to put buildTxt into header.scroll or footer.scroll to build text files for many pages at once.

I've loved this new feature in Scroll.

Plain text is a timeless way to read, review, and share your thoughts!

Hope you enjoy it as much as I have!

April 21, 2024 — The source code for this blog post contains a ScrollSet about the planets and generates this HTML file as well as a CSV, a TSV, and a JSON file. This page demonstrates ScrollSets.

ScrollSets are useful for small single day projects and large multi-year projects with thousands of concepts like PLDB (a Programming Language Database).

ScrollSets are normal plain text files written in Scroll that also contain measurements of concepts and output that data into formats ready for data visualization and analysis tools.

ScrollSets are line oriented but represent a table(s). You might call them deconstructed csvs or deconstructed spreadsheets.

• Use LLMs to instantly generate ScrollSets that are ready for human verification and improvement.
• Intermingle structured data with markup to annotate any and every part of a ScrollSets while still generating strict tabular files for data analysis tools.
• Put data, schema, citations, and documentation all in one (or more) plain text file(s) to easily share, collaborate on, and improve, all tracked by git for trust.
• Add unlimited citations (or none) to every measurement.

However, it's important to also have simple, lower tech, timeless tools and ScrollSets is one of those.

For YAML, just put your documentation and schema in YAML comments up top and then have a tiny script to read that YAML and dump CSV/TSV/JSON or whatever. YAML gives you loads of data structures to use and is widely supported in many languages. But generating HTML from the same file would require more work.

If you want to intermix markup content with your data, you can use Markdown to add the marked up content and then have code sections embedding the YAML and a tiny script to parse out those YAML blocks and write your data to disk.

If you don't like Scroll's (evolving) version and want to switch it will always be straightforward to automatically refactor to YAML.

ScrollSets evolved out of TrueBase. ScrollSets have eliminated the need for the TrueBase software (and existing TrueBase sites should be migrated to ScrollSets), but were informed by the TrueBase build experience.

Although ScrollSets are designed for a world with LLMs, the design is meant to be useful without them as well, and would also have been mildly useful 30 years ago.

The flat design was chosen for ergonomic reasons. ScrollSets seem like they might be useful enough to be worth breaking from Scroll convention a bit. Like all things in Scroll, ScrollSets are an experiment, and maybe this design will evolve.

Below is the ScrollSet embedded in this Scroll file.

May 8, 2023 — Some web apps are designed to load entire programs from a link.

But if a program contain certain characters, such as newlines, those links won't work as-is.

To create working links you have to run them through encodeURIComponent.

If you are creating blog content by hand this is annoying.

To update a program link you have to keep a copy of the program, update that, then run encodeURIComponent, then paste the result into your post.

Today's release of Scroll includes a small new addition that solves this problem. You can call them "Multline Links" or "Program Links".

Here is a program link to the Scroll web app.

If you click that link you should see "It worked!" in big green text.

Now let's show a complex real world example. The link below contains a 28 line program pasted verbatim from the Ohayo data science studio.

Discovery of the Elements

If you click that link you should see a few data visualizations generated from that program.

Edit the source and the link will update accordingly. Scroll handles the encodeURIComponent.

But by the HTML spec newlines are not a URL code point and generally are stripped and collapsed.

Program Links in Scroll handle encoding newlines and any other special characters so you can focus on your content and not the encoding.

April 13, 2023 — There's a tiny new symbol in Scroll today: !.

The exclamation mark stands for Counterpoint.

"Counters" aim to help writers strengthen their ideas by encouraging the integration of counterarguments throughout their essays.

You can write your own counters or invite counters from friends, LLMs or Internet commenters.

Counterpoints are easy to use: just start a line with !. For example:

You can go as deep as you want in countering your own counters!

• "Steel manning"
• "red teaming"
• "playing devil's advocate"

Countering is a helpful habit that helps clarify your thoughts, understand and empathize with other perspectives, fix logical blunders, hone your pitch, and improve at truth-seeking.

Counterpoints in Scroll are in a sense just a reminder to frequently use these techniques.

It can make your writing too verbose and lose reader interest.

Counters in Scroll give you a place to record every scattered counter as it comes to you.

You can then choose which are important enough to address in your main argument and relegate the rest to the source code (for the very avid reader).

Instead you only see them during write time.

For now, they are just an alternate syntax for comments.

But though they perform no new function, like traffic lines, sometimes form is function.

That being said, there are potentially interesting directions this could go in the future, and it would be interesting to hear ideas of what people would like in a v2.

Keep writing, keep thinking!

I am really happy with the current state of Scroll and so grateful for the many people who have helped us get it to this point. That being said, I had a feeling we were missing something important.

Introducing the blink tag. When you have something important to say and you want to interupt people's lives, use the blink tag. It's as easy as this:

Advertisers have known for ages that human eyes evolved to respond fast to motion. Now, using my new invention, the blink tag, you can bring this innovation from advertising to your products and websites.

Note: please don't use this. It's a joke

August 23, 2021 — Scroll is a new language and static site generator that is mostly written in Parsers which are both built on a new syntax called Particles.

In this demo, I extend Scroll by adding support for Markdown, Textile, and BBCode. I want to demonstrate how easy it is to compose many languages into one using Particles, which is due to what you might call the Indented or Off-side Heredoc pattern.

My implementation is still researchy—Parsers still needs a lot of work—but my main point here is to demo how simple and powerful indented heredocs are. They allow you to compose unlimited languages in one file in a clean and scalable way. You don't need escaping. I think this is a very helpful pattern you can use in your own languages and code.

Traditional heredocs use start and end delimiters, like these examples from Ruby and Python and Markdown:

In contrast here is the indented/off-side heredoc pattern in Scroll.

Emits this:

Markdown is best

No! textile is best

More extensive examples are pasted below. Source is here.

A simple but largely accurate description of Particles is that it is just indented or off-side HereDocs. Or in colloquial academic speak The Off-Side Rule is all you need. Most people learn of the off-side rule via Python. Imagine if your whole language was just the off-side rule. That's basically Particles. But my code is not the important thing—the important thing is this pattern. Try it yourself—I bet you will love it.