| Data Science with Scroll |
Data Science with Scroll ======================== by Breck Yunits https://twitter.com/breckyunits Breck Yunits January 6, 2025 A Tutorial ========== This tutorial will walk you through how to use Scroll for data analysis and visualization, from basic concepts to advanced techniques. What makes Scroll great for data science? ========================================= Scroll combines the simplicity of markdown-style syntax with powerful data transformation and visualization capabilities. You can: - Load data from various sources (CSV, JSON, etc.) - Transform and analyze data with simple commands - Create beautiful visualizations - Publish instantly using ScrollHub https://hub.scroll.pub/ ScrollHub - All in a simple, readable format Let's dive in! --- Part 1: Getting Started with Data ================================= Loading Sample Datasets ======================= Scroll comes with several sample datasets. Let's start with the famous iris dataset: sepal_length,sepal_width,petal_length,petal_width,species 6.1,3,4.9,1.8,virginica 5.6,2.7,4.2,1.3,versicolor 5.6,2.8,4.9,2,virginica 6.2,2.8,4.8,1.8,virginica 7.7,3.8,6.7,2.2,virginica 5.3,3.7,1.5,0.2,setosa 6.2,3.4,5.4,2.3,virginica 4.9,2.5,4.5,1.7,virginica 5.1,3.5,1.4,0.2,setosa 5,3.4,1.5,0.2,setosa You can also load datasets from Vega's collection: zip_code,latitude,longitude,city,state,county 501,40.922326,-72.637078,Holtsville,NY,Suffolk 544,40.922326,-72.637078,Holtsville,NY,Suffolk 601,18.165273,-66.722583,Adjuntas,PR,Adjuntas 602,18.393103,-67.180953,Aguada,PR,Aguada 603,18.455913,-67.14578,Aguadilla,PR,Aguadilla Basic Data Operations ===================== Let's explore some basic operations on the iris dataset: name,type,incompleteCount,uniqueCount,count,sum,median,mean,min,max,mode sepal_length,number,0,8,10,57.699999999999996,5.6,5.77,4.9,7.7,5.6 sepal_width,number,0,8,10,31.599999999999998,3.2,3.1599999999999997,2.5,3.8,2.8 petal_length,number,0,8,10,39.8,4.65,3.9799999999999995,1.4,6.7,4.9 petal_width,number,0,7,10,13.699999999999996,1.75,1.3699999999999997,0.2,2.3,0.2 species,string,0,3,10,null,null,,null,null,virginica This gives us summary statistics for each column. Let's look at filtering: sepal_length,sepal_width,petal_length,petal_width,species 5.3,3.7,1.5,0.2,setosa 5.1,3.5,1.4,0.2,setosa 5,3.4,1.5,0.2,setosa --- Part 2: Data Visualization ========================== Basic Plots =========== Let's start with a simple scatterplot of the iris data: Line Charts =========== Let's look at some time series data: Bar Charts ========== Let's create a bar chart showing precipitation: --- Part 3: Advanced Data Transformations ===================================== Grouping and Aggregation ======================== Let's look at some more complex transformations: weather,count,avg_max_temp,avg_min_temp drizzle,129,18.555813953488368,10.143410852713178 rain,459,15.535294117647041,9.04727668845315 sun,1674,18.064157706093184,8.87275985663083 snow,78,4.528205128205127,-1.4346153846153844 fog,582,15.261855670103111,8.527319587628869 Creating New Columns ==================== Let's add some computed columns: sepal_length,sepal_width,petal_length,petal_width,species,ratio 6.1,3,4.9,1.8,virginica,2.033333333333333 5.6,2.7,4.2,1.3,versicolor,2.074074074074074 6.2,2.8,4.8,1.8,virginica,2.2142857142857144 7.7,3.8,6.7,2.2,virginica,2.0263157894736845 --- Part 4: Advanced Visualizations =============================== Heatmaps ======== Let's create a heatmap of annual precipitation values: Multiple Views ============== You can create multiple visualizations: --- Conclusion ========== This tutorial covered the basics of data science with Scroll. Some key takeaways: - Scroll makes it easy to load and manipulate data - Visualizations are simple to create and customize - Complex transformations can be done with simple commands - Everything is readable and version-controllable ⁂ Built with Scroll v166.0.0 |
01/06/2025 |
632 |
3.2 |
| Scroll in 50 Questions |
Scroll in 50 Questions ====================== by Breck Yunits https://twitter.com/breckyunits Breck Yunits October 28, 2024 What is Scroll? =============== Scroll is a language for those who believe that what they cannot build with as few pieces as possible they do not understand. What is Scroll? =============== Scroll is a symbolic language. What is a symbolic language. ============================ A language you write with symbols. What is a symbol? ================= A symbol is a mark repeated to correlate to a pattern in nature. What is a mark? =============== A mark is a discoloration of a surface. What is a symbolic language? ============================ A symbolic language is a collection of symbols and symbol sequences that can be used to trigger the same electrical phenomena in the brain of an observer that seeing that pattern occur in nature would trigger. How do I use a symbolic language? ================================= By writing documents. What is a document? =================== It is a collection of marks on a surface. What is Particle Syntax? ======================== It is the syntax that Scroll uses. What is a syntax? ================= It is a set of rules for how to break up a document into symbols. What is a digital document? =========================== It is a file. What is a file? =============== It is a digital representation of a document as a series of bits. What is a bit? ============== A bit is a persistent location in space that has 2 states: a 1 or 0; a true or false; an on or off; a black or white; a hot or cold; a high or low. What is a character? ==================== A character is a 2D mark that maps to a unique fixed length bit sequence. What is a character encoding? ============================= A map of bit sequences to characters. What character encoding does Scroll use? ======================================== UTF8. For now. What is a word? =============== A word is a list of characters delimited by a word delimiter sequence. What is an atom? ================ An atom is another word for word in the Scroll language. What is a line? =============== A line is a sequence of atoms in the Scroll language delimited by a newline sequence. What is a Particle? =================== A particle in Scroll refers to an object that contains both atoms and a collection of particles referred to as the particle's subparticles. Are all particles also subparticles of a higher particle? ========================================================= Yes. What is a Parser? ================= A Parser is a particle that consumes certain other particles to perform transformations. What transformations can a Parser perform? ========================================== A Parser can perform a variety of transformations but the most common are to: - compile one particle in one language to another language. - format a particle into a standardized format - check a particle for errors - evalute the instructions in a particle and return the results - execute commands contained in the particle What is the syntax of Scroll? ============================= The syntax of Scrolls is Particle Syntax, or just Particles for short. What does Particle Syntax do? ============================= Partilces tells you how to divide a binary sequence into parts. What are the components of Particle Syntax? =========================================== - A sequence of binary bits is converted into characters using UTF8 - Atoms are sequences of characters separated by a single space. - Particles are a sequence of atoms separated by a newline. - Subparticles are indicated by a single indented space. How do I make a subparticle? ============================ - Just indent a particle by 1 space more than its parent particle. - For example, because I started this line with one space more than the line above, this line is a subparticle of that line. - And then this line, starting with 2 spaces, is a subparticle of the line above it. - But this line is a subparticle of the line above that starts with "Just" Can I use the Particles, Parsers, and Scroll concepts to design 2D or 3D languages that don't use computers at all? =================================================================================================================== Yes. But generally when we talk about PPS we are talking about the form designed for binary and computers. What is Parsers? ================ Parsers is a symbolic language where one defines Particles that can consume other Particles. It is a language for making other languages. What is Scroll? =============== Scroll is a language made out of Parsers. What is the purpose of the Parsers language? ============================================ The purpose of the Parsers Language is to make Scroll. What is the purpose of Scroll? ============================== The purpose of Scroll is to help humans communicate with humans and machines. How does one use Scroll to communicate? ======================================= By writing Scroll documents (aka Scroll Programs) to create things. What kinds of things is Scroll currently best at creating? ========================================================== Knowledge bases, blogs, websites, web pages, charts, animations, data science reports, ebooks, web forms, maps, that sort of thing. Why use Scroll and not just write HTML directly? ================================================ Scroll helps you better understand what you are writing about and anything HTML can make Scroll can make in fewer symbols. Scroll is a language for those who believe that what they cannot build with as few pieces as possible they do not understand. What is ScrollHub? ================== ScrollHub is a super server for creating and publishing websites instantly using Scroll. What is a cue? ============== Cue refers to the first atom in a particle. What is the significance of the cue? ==================================== When communicating you want to communicate the most important information first, so the cue is often the most important information. What is the cue used for? ========================= Most parsers use the cue to determine whether they should consume a particle or not. Can parsers use other ways to match against particles? ====================================================== Yes. What's another commmon way for parsers to match particles? ========================================================== Regular expressions. Can I build Scroll using only Particles and Parsers and no other languages? =========================================================================== Currently no. Parsers does not have enough functionality yet to be able to write parsers that provide everything Scroll needs. What other language do I currently need? ======================================== The primary implementation of Parsers currently is half Parsers, half Javascript. How does Parsers parse Scroll? ============================== The Parsers program is parsed and compiled by a Javascript implementation of Parsers which generates a Parser and compiler for Scroll programs. Do I need to master Javascript to use Parsers? ============================================== If you've mastered Javascript then using Parsers should be easy once you get the hang of it. Does Parsers have inheritance? ============================== Yes. Does Particle Syntax have types? ================================ No. Does Parsers have types? ======================== Yes. Parsers has atom types. Is Scroll like Lisp? ==================== Scroll and Parsers implement many of the best ideas from Lisp, and leave out a lot of unnecessary things (like the parens). Is Scroll designed for humans or AIs? ===================================== Both. ⁂ Built with Scroll v166.0.0 |
10/28/2024 |
1082 |
5.4 |
| Web Forms for Geniuses |
Web Forms for Geniuses ====================== by Breck Yunits https://twitter.com/breckyunits Breck Yunits October 1, 2024 If you want to make web forms the inefficient way, there are many other places to read how to do that. If you want to learn how geniuses are doing web forms, read on. * One Field at a Time is Retarded =============================== Building forms for one-field-at-a-time data entry is inefficient. People can't copy/paste your forms. E-receipts are a pain. Auto-form fill works poorly. It's a huge waste of your users' time that retards them and your business. And it will always be this way. Entire Form in One Field is Genius ================================== Everything becomes radically simpler. People can do things like copy/paste entire applications; email them; version control them; collaborate on them; all effortlessly. And guess what? You can still also compile to a one-field-at-a-time HTML form and allow users to swap back and forth! It's pure win, no loss. You can use this TODAY! ======================= This now ships in Scroll. Here is a test form. You can also see it in use in many of our products. https://scroll.pub/blog/../tests/forms.html a test form Because this is new technology, it is not for those that need spoon feeding. Believe it or not, making web forms also used to be complicated. (If you need this urgently, don't have time to figure it out yourself, and need to hire professional consulting, feel free to get in touch.) Don't whine. Build. =================== I'm tired of reading all the whiners commenting about how our CSS doesn't look so pretty yet, or the tiny little feature you need that isn't supported. I don't care much to hear from whining losers. I want to hear from winners. From builders. From those who strive for genius. Beautiful designs, autoincrement, auto timestamp, joins, decision trees, all that stuff is coming. The world is gonna love this. If you are a winner, a builder, a genius, join us on GitHub or Reddit. https://github.com/breck7/scroll GitHub https://www.reddit.com/r/WorldWideScroll/ Reddit ⁂ Built with Scroll v166.0.0 |
10/01/2024 |
369 |
1.8 |
| The Successor to RSS is Git Clone |
The Successor to RSS is Git Clone ================================= by Breck Yunits https://twitter.com/breckyunits Breck Yunits September 2, 2024 Today on HackerNews Kevin Damm had a great idea: put a `<link>` tag on blogs to point to the git Source Code Repository, much as blogs today have `<link>` tags pointing to their RSS feeds. https://news.ycombinator.com/item?id=41427044 great idea I've added this feature to Scroll and it's live now. If you View Source of this page, you'll see: https://scroll.pub/blog/../index.html Scroll ``` ``` Client applications can start looking for and taking advantage of these tags. * RSS was great, but it's no match for Git Clone ============================================== Git clone solves a million problems. You get a copy of the entire works of your favorite authors that is blazing fast, works offline, is searchable, transformable, censorship resistant, auditable, more trustworthy, ad free, tracker free, et cetera. There is no comparison to the old ways of doing things. * What do you think? ================== Have you already been doing this? Are there ways we can make it better? Would love to hear your feedback! ⁂ Notes ===== vladimyr on bluesky dug up a couple of cool examples of prior art. https://bsky.app/profile/vladimyr.bsky.social/post/3l3anievhnm2t vladimyr on bluesky Built with Scroll v166.0.0 |
09/02/2024 |
224 |
1.1 |
| A New Way to Program |
A New Way to Program ==================== by Breck Yunits https://twitter.com/breckyunits Breck Yunits August 30, 2024 Particles, Scroll, and the Parsers Programming Language ======================================================= I've recorded a short video (1 minute version; 10 minute version) about our recent work which has begun to eat the software world. https://www.youtube.com/watch?v=tpJY7PUmBIU 1 minute version https://www.youtube.com/embed/YcnWSNGEHvI?si=p_vOxpt9lusNKgli&t=75s 10 minute version * Outline ======= 1. Is this stuff relevant to you? ================================= Whether you are an experienced, aspiring, or casual programmer, yes. If you are a writer, yes. If you are a thinker, yes. If you are a builder, yes. If you choose to always remain illiterate to keep your brain different than the crowd, then no. (I applaud your brave creative choice and as a scientist I'm curious to see what happens!) 2. What is Particle Syntax (Particles)? ======================================= One liner: a syntax-free syntax for splitting files into particles (separated by line breaks) which can have atoms (separated by spaces) and subparticles (indented lines). Vibe https://particles.scroll.pub Vibe Playground https://sdk.scroll.pub/sandbox/ Playground LeetSheet https://scroll.pub/particlesLeetsheet.html LeetSheet [Image Omitted] width 400 Each line is a particle. Each dot is a atom. Particles lets you easily compare programs and see which one is simpler (less heavy!). 3. What is Scroll? ================== One liner: Scroll is a language for scientists of all ages where you write and combine particles (written in Particles) to evolve and publish your most intelligence ideas to HTML, PDFs, CSVs, JSON files, movie files, audio files, slideshows, charts, books, et cetera. Vibe https://scroll.pub/blog/index.html Vibe Playground https://try.scroll.pub/ Playground LeetSheet https://scroll.pub/leetsheet.html LeetSheet [Image Omitted] width 400 Scroll is a collection of Parsers that allow you to evolve and publish your most intelligent ideas. 4. What is Parsers? =================== One liner: Parsers is a programming language where programmers write parsers (written in Particles) which consume particles (written in Particles) and also contain logic for translating those particles into actions on computing machines. Vibe https://github.com/breck7/scroll/tree/main/parsers Vibe Playground https://sdk.scroll.pub/designer#url%20https%3A%2F%2Fscroll.pub%2Fscroll.parsers Playground LeetSheet https://scroll.pub/parserLeetsheet.html LeetSheet [Image Omitted] width 400 Programmers who understand how computing machines work write Parsers to translate particles into executable machine code. 5. How do you get started? ========================== ScrollHub http://hub.scroll.pub/ ScrollHub Subreddit https://www.reddit.com/r/WorldWideScroll/ Subreddit ⁂ Built with Scroll v166.0.0 |
08/30/2024 |
425 |
2.1 |
| Tables: a microlang for data science |
Tables: a microlang for data science ==================================== by Breck Yunits https://twitter.com/breckyunits Breck Yunits August 9, 2024 Tables, aka spreadsheets, are arguably the most important _visual_ thought tool. But no one has designed the perfect _textual_ language for manipulating them. Until now. Say hello to Tables =================== Try now https://try.scroll.pub/#scroll%0A%20theme%20gazette%0A%20%0A%20expander%20Presidents%0A%20https%3A%2F%2Fohayo.breckyunits.com%2Fohayo%2Fpackages%2Fsamples%2Fpresidents.csv%0A%20%20groupBy%20Party%0A%20%20%20orderBy%20-count%0A%20%20%20%20printTable%0A%20%20scatterplot%0A%20%20%20x%20number%0A%20%20%20y%20Party%0A%20%20%20fill%20Party%0A%20%0A%20expander%20Elements%0A%20https%3A%2F%2Fohayo.breckyunits.com%2Fohayo%2Fpackages%2Fsamples%2Fperiodic-table.csv%0A%20%20scatterplot%0A%20%20%20x%20Year%0A%20%20%20y%20AtomicNumber%0A%20%20%20fill%20Radioactive%0A%20%20%20symbol%20Metal%0A%20%0A%20%2F%2F%20expander%20Inline%20Data%0A%20table%0A%20%20scatterplot%0A%20%20%20title%20Top%2050%20programming%20language%20projects%20on%20GitHub.%0A%20%20%20x%20appeared%0A%20%20%20y%20rank%0A%20%20%20radius%20githubStars%0A%20%20%20label%20name%0A%20%20orderBy%20appeared%0A%20%20%20sparkline%0A%20%20%20%20y%20githubStars%0A%20%20%20%20color%20green%0A%20%20%20select%20name%20githubStars%0A%20%20%20%20heatrix%0A%20%20data%0A%20%20%20rank%2Cname%2Cappeared%2CgithubStars%0A%20%20%203%2CPython%2C1991%2C61378%0A%20%20%204%2CJava%2C1995%2C19037%0A%20%20%208%2CPerl%2C1987%2C1886%0A%20%20%209%2CRuby%2C1995%2C21790%0A%20%20%2010%2CPHP%2C1995%2C37688%0A%20%20%2013%2CGo%2C2009%2C121611%0A%20%20%2014%2CTypeScript%2C2012%2C99410%0A%20%20%2018%2CPowerShell%2C2006%2C44220%0A%20%20%2020%2CRust%2C2010%2C95455%0A%20%20%2021%2CSwift%2C2014%2C67064%0A%20%20%2025%2CKotlin%2C2011%2C48462%0A%20%20%2027%2CClojure%2C2007%2C10392%0A%20%20%2028%2CCoffeeScript%2C2009%2C16466%0A%20%20%2029%2CElixir%2C2011%2C24093%0A%20%20%2030%2CErlang%2C1986%2C11203%0A%20%20%2034%2CJulia%2C2012%2C45137%0A%20%20%2041%2CRacket%2C1994%2C4741%0A%20%20%2042%2CSolidity%2C2014%2C22794%0A%20%20%2043%2CWebAssembly%2C2015%2C6621%0A%20%20%2045%2CZig%2C2015%2C32864%0A%20%20%2047%2CNim%2C2008%2C16308%0A%20%20%2050%2CTOML%2C2013%2C19332 Try now ⁂ Acknowledgements ================ Tables evolved over the past decade primarily by: - just doing what Hadley Wickham did in dplyr https://dplyr.tidyverse.org/ dplyr https://hadley.nz/ Hadley Wickham - using Mike Bostock's d3 under the hood https://bost.ocks.org/mike/ Mike Bostock https://d3js.org/ d3 - reading all of Jeffrey Heer's papers https://homes.cs.washington.edu/~jheer/ Jeffrey Heer - thinking about what would be the best tool for Max Roser and Hannah Ritchie. https://ourworldindata.org/ Max Roser and Hannah Ritchie Built with Scroll v166.0.0 |
08/09/2024 |
479 |
2.4 |
| Build Your Own Counters With 3 Words |
Build Your Own Counters With 3 Words ==================================== by Breck Yunits https://twitter.com/breckyunits Breck Yunits August 2, 2024 August 2, 2024 I like collections of interesting numbers, such as BioNumbers. Animated counters provide a useful perspective when pondering large numbers. I wanted to make it as easy as possible for anyone to create these counters. https://bionumbers.hms.harvard.edu/Search.aspx?task=searchbypop BioNumbers So, today I added a new parser to Scroll: *counter*. ``` counter 1.1 Heartbeats ``` * Try now https://try.scroll.pub/#scroll%0A%20theme%20gazette%0A%20%0A%20%23%20Show%20HN%3A%20Make%20a%20counter%20with%203%20words%0A%20%0A%20New%20in%20Scroll%20v112.1.0.%20Read%20more.%0A%20%20https%3A%2F%2Fscroll.pub%2Fblog%2Fcounters.html%20Read%20more.%0A%20%0A%20code%0A%20%20counter%20%5BRate%2FSecond%5D%20%5BDescription%5D%0A%20%0A%20%23%20Since%20you%20loaded%20this%20page...%0A%20%0A%20thinColumns%0A%20%0A%20%23%23%20World%20Population%0A%20counter%204.5%20Babies%20Born%0A%20%20https%3A%2F%2Fourworldindata.org%2Fbirths-and-deaths%0A%20counter%201.7%20Human%20Deaths%0A%20%20https%3A%2F%2Fourworldindata.org%2Fbirths-and-deaths%0A%20counter%208123456789%20heart%20beats%0A%20%20https%3A%2F%2Fourworldindata.org%2Fgrapher%2Fpopulation%0A%20%0A%20%23%23%20In%20Your%20Body%0A%20counter%203112123%20Red%20Blood%20Cells%20Made%0A%20%20https%3A%2F%2Fwww.ncbi.nlm.nih.gov%2Fbooks%2FNBK2263%2F%0A%20counter%203812123%20Cells%20Made%0A%20%20https%3A%2F%2Fwww.ncbi.nlm.nih.gov%2Fbooks%2FNBK2263%2F%0A%20counter%202123456789123456789123%20ATP%20molecules%20made%0A%20%20https%3A%2F%2Fbio.libretexts.org%2FBookshelves%2FHuman_Biology%2FHuman_Biology_(Wakim_and_Grewal)%2F05%253A_Cells%2F5.09%253A_Cellular_Respiration%0A%20%0A%20%23%23%20In%20Your%20Brain%0A%20counter%201.2e12%20Action%20Potentials%20Fired%0A%20%20https%3A%2F%2Fwww.khanacademy.org%2Ftest-prep%2Fmcat%2Forgan-systems%2Fneural-synapses%2Fa%2Fsignal-propagation-the-movement-of-signals-between-neurons%0A%20counter%203.64e18%20Ketones%20Consumed%20(Ketosis)%0A%20%20https%3A%2F%2Fwww.ncbi.nlm.nih.gov%2Fpmc%2Farticles%2FPMC7699472%2F%0A%20counter%204.64e18%20Glucose%20Molecules%20Consumed%0A%20%20https%3A%2F%2Fwww.ncbi.nlm.nih.gov%2Fpmc%2Farticles%2FPMC3900881%2F%0A%20%0A%20%23%23%20World%20Economy%0A%20counter%202.5%20Cars%20Made%0A%20%20https%3A%2F%2Fwww.statista.com%2Fstatistics%2F262747%2Fworldwide-automobile-production-since-2000%2F%0A%20counter%2026012%20Cups%20of%20Coffee%20Consumed%0A%20%20https%3A%2F%2Fwww.rgare.com%2Fknowledge-center%2Farticle%2Fcoffee-consumption-how-much-is-too-much-and-how-little-is-not-enough%0A%20counter%2059321%20YouTubes%20Watched%0A%20%20https%3A%2F%2Fthesocialshepherd.com%2Fblog%2Fyoutube-statistics%0A%20%0A%20%23%23%20Flight%0A%20counter%200.00008%20Objects%20Launched%20into%20Space%0A%20%20https%3A%2F%2Fourworldindata.org%2Fgrapher%2Fyearly-number-of-objects-launched-into-outer-space%0A%20counter%200.00004%20Planes%20Built%0A%20%20https%3A%2F%2Fwww.ibisworld.com%2Fglobal%2Fmarket-research-reports%2Fglobal-commercial-aircraft-manufacturing-industry%2F%0A%20counter%201.17%20Planes%20Landed%0A%20%20https%3A%2F%2Fwww.faa.gov%2Fair_traffic%2Fby_the_numbers%0A%20%0A%20%23%23%20Animals%0A%20counter%200.00004%20Blue%20Whales%20Born%0A%20%20https%3A%2F%2Fwww.marinemammalcenter.org%2Fanimal-care%2Flearn-about-marine-mammals%2Fcetaceans%2Fblue-whale%0A%20counter%200.001%20Elephants%20Born%0A%20%20https%3A%2F%2Felephantconservation.org%2Fabout-elephants%2F25-things-to-know%2F%0A%20counter%203.1e10%20Ants%20Born%0A%20%20https%3A%2F%2Fprivateexterminator.com%2Fare-all-the-ants-as-heavy-as-all-the-humans%0A%20 Try now =================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================== * Since you loaded this page... ============================= World Population ================ 4.5 Babies Born https://ourworldindata.org/births-and-deaths 4.5 Babies Born 1.7 Human Deaths https://ourworldindata.org/births-and-deaths 1.7 Human Deaths 8123456789 heart beats https://ourworldindata.org/grapher/population 8123456789 heart beats In Your Body ============ 3112123 Red Blood Cells Made https://www.ncbi.nlm.nih.gov/books/NBK2263/ 3112123 Red Blood Cells Made 3812123 Cells Made https://www.ncbi.nlm.nih.gov/books/NBK2263/ 3812123 Cells Made 2123456789123456789123 ATP molecules made https://bio.libretexts.org/Bookshelves/Human_Biology/Human_Biology_(Wakim_and_Grewal)/05%3A_Cells/5.09%3A_Cellular_Respiration 2123456789123456789123 ATP molecules made In Your Brain ============= 1.2e12 Action Potentials Fired https://www.khanacademy.org/test-prep/mcat/organ-systems/neural-synapses/a/signal-propagation-the-movement-of-signals-between-neurons 1.2e12 Action Potentials Fired 3.64e18 Ketones Consumed (Ketosis) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7699472/ 3.64e18 Ketones Consumed (Ketosis) 4.64e18 Glucose Molecules Consumed https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3900881/ 4.64e18 Glucose Molecules Consumed World Economy ============= 2.5 Cars Made https://www.statista.com/statistics/262747/worldwide-automobile-production-since-2000/ 2.5 Cars Made 26012 Cups of Coffee Consumed https://www.rgare.com/knowledge-center/article/coffee-consumption-how-much-is-too-much-and-how-little-is-not-enough 26012 Cups of Coffee Consumed 59321 YouTubes Watched https://thesocialshepherd.com/blog/youtube-statistics 59321 YouTubes Watched Flight ====== 0.00008 Objects Launched into Space https://ourworldindata.org/grapher/yearly-number-of-objects-launched-into-outer-space 0.00008 Objects Launched into Space 0.00004 Planes Built https://www.ibisworld.com/global/market-research-reports/global-commercial-aircraft-manufacturing-industry/ 0.00004 Planes Built 1.17 Planes Landed https://www.faa.gov/air_traffic/by_the_numbers 1.17 Planes Landed Animals ======= 0.00004 Blue Whales Born https://www.marinemammalcenter.org/animal-care/learn-about-marine-mammals/cetaceans/blue-whale 0.00004 Blue Whales Born 0.001 Elephants Born https://elephantconservation.org/about-elephants/25-things-to-know/ 0.001 Elephants Born 3.1e10 Ants Born https://privateexterminator.com/are-all-the-ants-as-heavy-as-all-the-humans 3.1e10 Ants Born Computer Chips ============== 66.7 CPUS made by Intel https://www.intel.com/content/www/us/en/architecture-and-technology/global-manufacturing.html 66.7 CPUS made by Intel 6234556543 Transistors made by Nvidia https://nvidianews.nvidia.com/news/nvidia-announces-hopper-architecture-the-next-generation-of-accelerated-computing 6234556543 Transistors made by Nvidia 61234556543 Transistors made by Intel https://www.intel.com/content/www/us/en/history/museum-transistors-to-transformations-brochure.html 61234556543 Transistors made by Intel ⁂ Built with Scroll v166.0.0 |
08/02/2024 |
1029 |
5.1 |
| Heatrix: Heat Maps + Matrix Visualizations |
Heatrix: Heat Maps + Matrix Visualizations ========================================== by Breck Yunits https://twitter.com/breckyunits Breck Yunits July 28, 2024 [Image Omitted] A heatrix on CancerDB. https://cancerdb.com/blog/cancer-heatmaps.html A heatrix on CancerDB. July 28, 2024 Do you love those GitHub Activity Charts and want to make your own but don't know what they are called or what library to use and your work is busy and you have kids and sorry I'll call you back someone is crying? Introducing Scroll's newest parser: `heatrix`! https://scroll.pub/blog/../index.html Scroll's Try Now | Watch Video https://try.scroll.pub/#scroll%0A%20%23%20Heatrix%20Demos%0A%20%0A%20%23%23%20Adjust%20the%20height%2Fwidth%20of%20individual%20cells%2C%20columns%2C%20or%20rows%3A%0A%20heatrix%0A%20%20h20%3BScroll%20h20%3BPLDB%0A%20%20400%20700%0A%20%0A%20%23%23%20Add%20links%3A%0A%20heatrix%0A%20%20h20%3BScroll%3Bhttps%3A%2F%2Fscroll.pub%20h20%3BPLDB%3Bhttps%3A%2F%2Fpldb.io%0A%20%20400%20700%0A%20%0A%20%23%23%20Use%20numbers%20as%20labels%3A%0A%20heatrix%0A%20%20'2021%20'2017%0A%20%20400%20700%0A%20%0A%20%23%23%20Use%20your%20own%20custom%20color%20palettes%20and%20set%20your%20own%20thresholds%3A%0A%20heatrixAdvanced%0A%20%20table%0A%20%20%20%20%0A%20%20%20h10%3Bw30%20'2015%20'2016%20'2017%20'2018%20'2019%20'2020%20'2021%20'2022%20'2023%20'2024%0A%20%20%20h30%3Bw30%3B%200%200%205%201%202%2011%2015%2010%2012%2056%0A%20%20scale%0A%20%20%20%23ffebee%200%0A%20%20%20%23ffcdd2%201%0A%20%20%20%23ef9a9a%205%0A%20%20%20%23e57373%2010%0A%20%20%20%23ef5350%2025%0A%20%20%20%23b71c1c%2050 Try Now https://www.youtube.com/watch?v=ycF3k63qn1Y Watch Video ========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================= `heatrix` let's you craft custom heatmap visualizations with the fewest keystrokes possible. Heat Map + Matrix = Heatrix =========================== [Image Omitted] Source code. Gif made with CleanShotX. https://try.scroll.pub/#scroll%0A%20heatrix%0A%20%208%206%207%205%0A%20%2030%209 Source code https://cleanshot.com/ CleanShotX * Not just a library, it's a microlanguage! ========================================= `heatrix` is a microlang that let's you customize your visualizations using "directives": Adjust the height/width of individual cells, columns, or rows: ============================================================== Add links: ========== Add labels: =========== Use numbers as labels: ====================== Use your own custom color palettes and set your own thresholds: =============================================================== * Like Heatrix? There's more where that came from! ================================================ Join the World Wide Scroll Beta today! ====================================== If you like `heatrix` and want to see more stuff like this, but a folder on the World Wide Scroll beta and good things will come your way: https://wws.scroll.pub/ World Wide Scroll beta ⁂ Built with Scroll v166.0.0 |
07/28/2024 |
470 |
2.4 |
| Contacts: A Microlang for Managing Contacts |
Contacts: A Microlang for Managing Contacts =========================================== by Breck Yunits https://twitter.com/breckyunits Breck Yunits July 16, 2024 July 16, 2024 Imagine a single plain text file named `contacts.scroll` where you stored the contact info for your family, friends, and colleagues, and you could track changes with git, compile it to a beautiful PDF or HTML page with search and sort, and it would also compile to CSV and/or JSON for import into iPhone, Android, Gmail, Outlook, et cetera? Also imagine that this tool is free, open source, and public domain. * Introducing *Contacts*, a microlanguage (currently 23 lines of code, including comments) that makes it as concise as possible to add, prune, display, print and export your contacts. https://github.com/breck7/scroll/blob/main/microlangs/contacts.parsers 23 lines of code * There's almost nothing to learn (the code is self-explanatory). In fact, it's almost guaranteed that using Contacts you will need to write less (we've taken every unnecessary character out). * Example ======= ``` 👤 Jack Doe phone +1 (555) 123-4567 email john.doe@example.com birthday 2/23/84 notes Daughter - Samantha. 👤 Jill Smith phone +1 (555) 123-4562 email jill@gmail.com birthday 1/23/80 ``` * How to use ========== 1. Create a file named something like `contacts.scroll` with this content: ``` // You can import the file, or just copy/paste the 21 lines of code here. [pathToScroll]/microlangs/contacts.parser title My Contacts buildConcepts contacts.csv contacts.json contacts.tsv buildHtml theme gazette mediumColumns 1 printTitle table printTable tableSearch 👤 Jack Doe phone +1 (555) 123-4567 email john.doe@example.com birthday 2/23/84 notes Daughter - Samantha. 👤 Jill Smith phone +1 (555) 123-4562 email jill@gmail.com birthday 1/23/80 ``` 2. Run `scroll build` Done! You should now see a contacts.html file for display and printing as well as TSV, CSV, and JSON files. https://scroll.pub/blog/../tests/contacts.html contacts.html * [Image Omitted] Contacts is interoperable with pen and paper. * What do you think? Anything that should be added/removed/improved? * Notes ===== 1. The idea for Contacts was sparked by this tweet from Tyler Tringas. https://x.com/tylertringas/status/1813239718957461528 this tweet from Tyler Tringas ⁂ Built with Scroll v166.0.0 |
07/16/2024 |
374 |
1.9 |
| How I Designed Helpful 404's and Stopped Worrying about Broken Links |
How I Designed Helpful 404's and Stopped Worrying about Broken Links ==================================================================== by Breck Yunits https://twitter.com/breckyunits Breck Yunits July 12, 2024 _Woohoo!_ Your blog is on top of Reddit! _OH NO._ There's a typo in your url. 🤦 *What do you do?* * You try to calm yourself down. _Almost no one looks at the url and the content of the article is what's important and the traffic is coming_ But then you think _all I can see is that i before e after c!_ But then you think _is that *really* a rule_ and _maybe that's how it's spelled in London?_ * You can't help yourself. It's an affront to your craftsmanship to not correct the bad filename. You know you can so easily rename the file but then existing links will break and so you'll have to add redirects but that means more files to maintain or god forbid that would require adding a server and you just love your static blog and it's almost 2pm and you have to leave and pick up the kids from school what do you do!?!! Or do you just break the links and going forward everyone will have the proper URL but everyone who clicks the old links will see an unhelpful 404 and you reflect on how great this thing that Tim Berners-Lee gave to the world is and how could you litter his creation with unhelpful 404s? * If the story above resonates with you, the bad news is you may suffer from Broken Link Phobia. Luckily, there is now a cure. * Introducing: Helpful 404's - never worry about broken links again! ================================================================== Scroll now comes with a way to generate a very helpful 404 page so your visitors will get redirected to the right place even if the URL they clicked is off by a few characters. The best part? It works entirely client side on static sites. How to use ========== *Step 1* Create a sitemap: ```sitemap.scroll buildTxt sitemap.txt baseUrl https://scroll.pub/ printSiteMap ``` *Step 2* Create a 404 page: ```404.scroll buildHtml Sorry, the url you requested was not found. helpfulNotFound sitemap.txt ``` That's it! The `helpfulNotFound` parser in Scroll will add Javascript to the 404 page that fetches all the urls found in the sitemap and shows the user the closest match. As a bonus, you also get a `sitemap.txt` file great for Google SEO and other uses. See it in action here by clicking on this intentionally broken link: https://scroll.pub/blog/helpful_404s.html intentionally broken link * I've been using Helpful 404s on all my sites for the past couple of months and it's been delightful. I no longer spend any time worrying about making improvements to URLS. I just do it and I know that users following old links will still have a good experience. * If you like stuff like Helpful 404's, there's _a lot_ more like that in Scroll, which is public domain and open source. Try it today, and if you like what we are doing please consider bringing your site to the World Wide Scroll! https://wws.scroll.pub World Wide Scroll ⁂ Built with Scroll v166.0.0 |
07/12/2024 |
547 |
2.7 |
| Convert CSVs to ScrollSets |
Convert CSVs to ScrollSets ========================== by Breck Yunits https://twitter.com/breckyunits Breck Yunits July 9, 2024 ScrollSets are _very_ useful once you get going. But starting from scratch could be a bit tedious. https://breckyunits.com/scrollsets.html ScrollSets Not anymore! Just drop your CSV or TSV into the textarea below to generate a ScrollSet. (P.S. using LLMs with ScrollSets is another great way to get started) https://sets.scroll.pub/ using LLMs with ScrollSets * Your Data: ========== ScrollSet: ========== Related ======= ScrollSets: source code for CSVs ================================ 04/21/2024 https://scroll.pub/blog/scrollsets.html Built with Scroll v166.0.0 |
07/09/2024 |
100 |
0.5 |
| Changes: A Microlang for Change Logs |
Changes: A Microlang for Change Logs ==================================== by Breck Yunits https://twitter.com/breckyunits Breck Yunits [Image Omitted] July 1, 2024 Regardless of what languages you write your programs in, you probably maintain files like _changeLog.txt_ or _releaseNotes.txt_. Change logs are very helpful for: - informing your users of new features and bug fixes - providing detailed technical information to your development team (and open source contributors) - periodical data analysis for strategic project planning - how fast are we shipping new features? fixing bugs? breaking things? What if there was 1 language that let you do all 3 things at once? * Introducing Changes, a microlang (only 70 lines of code, including comments) that makes it as concise as possible to write, read and analyze change logs. https://github.com/breck7/scroll/blob/main/microlangs/changes.parsers 70 lines of code Changes generates a pretty HTML file for your end users; allows for including unlimited detailed technical information for your developers; and generates summary statistics as TSV, CSV, and JSON for your project managers. * There's almost nothing to learn (the code is self explanatory). In fact, it's almost guaranteed that using Changes you will need to write less (we've taken every unnecessary character out). * Example ======= We use Changes for the Scroll Release Notes. https://scroll.pub/blog/../releaseNotes.html Scroll Release Notes Be sure to check out the source code that generates that HTML. https://github.com/breck7/scroll/blob/main/releaseNotes.scroll check out the source code * How to use ========== 1. Install Scroll ``` npm install -g scroll-cli ``` 2. Create a release notes file: ```releaseNotes.scroll title My Release Notes printTitle buildConcepts releaseNotes.csv releaseNotes.json releaseNotes.tsv // You can import the changes parsers or just copy/paste the 70 lines into your own file. [pathToScroll]/microlangs/changes.parser thinColumns 📦 0.1.1 7/1/2024 🏥 fixed bug in command line app 📦 0.1.0 7/1/2024 🎉 added command line app endColumns ``` 3. Run `scroll build` Done! You should now see a `releaseNotes.html` file as well as a TSV, CSV, and JSON file. * What do you think? Anything that should be added/removed/improved? ⁂ Related ======= - Changelog https://en.wikipedia.org/wiki/Changelog Changelog - Release Notes https://en.wikipedia.org/wiki/Release_notes Release Notes - Major Version Numbers are Not Sacred https://tom.preston-werner.com/2022/05/23/major-version-numbers-are-not-sacred Major Version Numbers are Not Sacred - What can we learn from programming language version numbers? https://breckyunits.com/versionNumbers.html What can we learn from programming language version numbers? Built with Scroll v166.0.0 |
07/01/2024 |
431 |
2.2 |
| Write scatterplot to get a scatterplot |
Write scatterplot to get a scatterplot ====================================== by Breck Yunits https://twitter.com/breckyunits Breck Yunits June 24, 2024 I _engineered on_ many innovative data science tools, including Grapher and Ohayo^engineering. https://github.com/breck7/ohayo Ohayo https://github.com/owid/owid-grapher Grapher So I was excited to see Observable launch something new: Plot. https://observablehq.com/plot/ Plot https://observablehq.com/ Observable * Plot is already great on its own. It handles many of the hard parts of data vis, and it's open source, so we can help them fix the parts that are still a little rough. https://github.com/observablehq/plot open source But I found a way to take Plot to the next level: I combined it with Scroll. https://scroll.pub/blog/../index.html Scroll The Plot/Scroll integration is early, but I can already tell it will evolve into a _very useful_ data science tool. * A Scatterplot with 2 words ========================== You can now make scatterplots with 2 words. Obviously you want to add a few more words, but I don't think I'm exaggerating when I say that this is the simplest way to generate a scatterplot now and forever will be (until the day when we have mind-reading machines that can generate a scatterplot with zero words). But enough talk, let's see some code. A tutorial is below. To follow along, create a new website in 1 second using ScrollHub or install Scroll locally with `npm install -g scroll-cli`. http://hub.scroll.pub ScrollHub * Step 1: Generate a scatterplot with 2 words =========================================== * Step 2: Add a title =================== * Step 3: Add inline data ======================= * Step 4: Everything everywhere all at once ========================================= * That's it (for now)! If you like where this is headed, give us a star on GitHub. https://github.com/breck7/scroll give us a star on GitHub ⁂ ^engineering: The difference between programming and engineering? It takes programming to make a program that works. It takes engineering to make a program that _barely_ works. Built with Scroll v166.0.0 |
06/24/2024 |
342 |
1.7 |
| Stamp: a microlang for project templates |
Stamp: a microlang for project templates ======================================== by Breck Yunits https://twitter.com/breckyunits Breck Yunits June 23, 2024 Regardless if you specialize in React, Rails, Django, Next, Java, C#, or ObjectiveC, you probably use _templates_ to start new projects. Templates generate a handful of files like `readme.md`, `.gitignore`, and `main`. They also initialize a handful of directories like `src/` and `tests/`. What if it was even easier to make, edit and use these templates? * Introducing Stamp, a microlang (only 60 lines of code, including comments) that makes it as concise as possible to write, edit, share and expand project templates. https://github.com/breck7/scroll/blob/main/parsers/stamp.parsers 60 lines of code * How to use ========== 1. Install Scroll ``` npm install -g scroll-cli ``` 2. Create `myFirstStamp.scroll` ``` stamp .gitignore *.html readme.scroll # My First Stamp scripts/ hello.js console.log("Hello world") ``` 3. Run `scroll build` Done! * Stamp was jointly created by me and Guillaume Papin. Other tools (listed below) do similar things, but with some slight differences. https://github.com/Sarcasm Guillaume Papin https://github.com/breck7/scrollsdk/issues/120 jointly created ⁂ [Image Omitted] Prior Art ========= - cookiecutter https://github.com/cookiecutter/cookiecutter cookiecutter - Dired https://www.gnu.org/software/emacs/manual/html_node/emacs/Dired.html Dired - vimv https://github.com/thameera/vimv vimv - ptar https://github.com/jtvaughan/ptar ptar - shar https://en.wikipedia.org/wiki/Shar shar - dar http://dar.linux.free.fr/ dar - metatar https://github.com/xyproto/metatar metatar - txtar https://pkg.go.dev/golang.org/x/tools@v0.1.0/txtar txtar - Archiving files in plain text https://blog.oquijano.net/id/9?lang=en Archiving files in plain text - Wikipedia list of archive formats https://en.wikipedia.org/wiki/List_of_archive_formats Wikipedia list of archive formats Built with Scroll v166.0.0 |
06/23/2024 |
300 |
1.5 |
| Scroll builds text files |
Scroll builds text files ======================== by Breck Yunits https://breckyunits.com Breck Yunits HTML | TXT link textFiles.html HTML link textFiles.txt TXT April 29, 2024 Scroll is a new language for building HTML and CSV files that powers blogs, websites and knowledge bases. https://scroll.pub/blog/scrollsets.html knowledge bases https://scroll.pub/blog/../index.html Scroll * I often need plain text outputs in addition to HTML files. I used to make those plain text versions copying and pasting text by hand. Now Scroll has plain text output built-in. * How do I tell Scroll to build plain text files? =============================================== Add a single line: `buildTxt` ```example.scroll buildTxt Hello world. ``` `scroll build` will create `example.txt`. Done! * You may want to put `buildTxt` into `header.scroll` or `footer.scroll` to build text files for many pages at once. * I've *loved* this new feature in Scroll. Plain text is a *timeless* way to read, review, and share your thoughts! Hope you enjoy it as much as I have! ⁂ Built with Scroll v166.0.0 |
04/29/2024 |
175 |
0.9 |
| ScrollSets: source code for CSVs |
ScrollSets: source code for CSVs ================================ by Breck Yunits https://breckyunits.com Breck Yunits [Image Omitted] More examples of ScrollSets from sets.scroll.pub. https://sets.scroll.pub/ sets.scroll.pub April 21, 2024 The source code for this blog post contains a ScrollSet about the planets and generates this HTML file as well as a CSV, a TSV, and a JSON file. This page demonstrates *ScrollSets*. https://scroll.pub/blog/scrollsets.scroll source code for this blog post https://scroll.pub/blog/planets.csv CSV https://scroll.pub/blog/planets.tsv TSV https://scroll.pub/blog/planets.json JSON ScrollSets are useful for small single day projects and large multi-year projects with thousands of concepts like PLDB (a Programming Language Database). https://pldb.io PLDB * ScrollSets are normal plain text files written in Scroll that also contain measurements of concepts and output that data into formats ready for data visualization and analysis tools. https://scroll.pub/ Scroll ScrollSets are line oriented but represent a table(s). You might call them _deconstructed csvs_ or _deconstructed spreadsheets_. - Use LLMs to *instantly generate ScrollSets* that are ready for human verification and improvement. - Intermingle structured data with markup to *annotate any and every part of a ScrollSets* while still generating strict tabular files for data analysis tools. - Put data, schema, citations, and documentation *all* in one (or more) plain text file(s) to easily share, collaborate on, and improve, all *tracked by git for trust*. - Add unlimited citations (or none) to *every* measurement. Quick Code Example: =================== ```planets.scroll This ScrollSets has 2 measures (columns) and 2 concepts (rows). Documentation, column definitions, rows and *any notes/markup/content* can go in the same file. # Measures (aka Header, aka Columns, aka Schema) idParser // Every concept needs an "id" (or other concept delimiter) extends abstractIdParser moonsParser extends abstractIntegerParser # Concepts (aka Rows) id mars moons 2 // I verified moon count with Google. - BY id jupiter moons 63 // Note: the moons of Jupiter have their own Wikipedia Page https://en.wikipedia.org/wiki/Moons_of_Jupiter moons of Jupiter buildConcepts demo.csv ``` The code above generates an HTML page and this: =============================================== ```demo.csv id,moons mars,2 jupiter,63 ``` Overview: ========= - ScrollSets are built from 4 atomic elements: - concepts - think of rows in a spreadsheet - denoted by a line starting with `id ` - concepts are multiple lines of measurements - measures - think of these as the column names in a spreadsheet, along with meta information about the column - aka "parsers" - measures are defined in Parsers that start with a line like `moonsParser` - values - these are just the values of the measurements - measurements - concept & measure & value = measurement - 1 measurement = 1 line - measurements can have nested comments that are stripped when compiling to TSV/CSV How to use ========== - A concept is like a row in a database. All concepts need an id (or other concept delimiter). When you write `id [conceptId]`, Scroll knows that is the beginning of a new concept. - Measure definitions (aka "parsers") must come before the first concept and are written as Parsers, just like any other Scroll Parser. Measure parsers need to extend one of the abstract measure parser classes defined in `measures.scroll`. - Measurements are then done like this `appeared 2024` FAQ === Isn't the better idea to enhance existing spreadsheet GUIs with LLM generation capabilities? ============================================================================================ Almost certainly. Using ScrollSets will be much slower and worse than future spreadsheet apps with carefully crafted LLM integrations. However, it's important to also have simple, lower tech, timeless tools and ScrollSets is one of those. Can't you do this same thing with YAML and/or Markdown? ======================================================= Yes! You can easily achieve the same thing as LLMs & ScrollSets using LLMs & YAML, or LLMs & YAML & Markdown. https://yaml.org/ YAML https://github.github.com/gfm/ Markdown For YAML, just put your documentation and schema in YAML comments up top and then have a tiny script to read that YAML and dump CSV/TSV/JSON or whatever. YAML gives you loads of data structures to use and is widely supported in many languages. But generating HTML from the same file would require more work. If you want to intermix markup content with your data, you can use Markdown to add the marked up content and then have code sections embedding the YAML and a tiny script to parse out those YAML blocks and write your data to disk. So, why use Scroll for storing data instead of YAML? ==================================================== Either can do the job. I expect the Scroll design to end up being more ergonomic, but that might not be true or may be unimportant. If you don't like Scroll's (evolving) version and want to switch it will always be straightforward to automatically refactor to YAML. What other related work is out there? ===================================== This is a simple pattern to implement, so I'm sure it is likely it has been done a few times before. Please let me know so I can include links to--and learn from--any other prior art. What are the advanced features? =============================== - Types correctly exported in JSON - Supports nested measures - Support for computed measures - Autojoins across files on ids^roadmap - Auto generates normalized tables for array measures^roadmap - Support for text blobs^roadmap ^roadmap: Planned. What is the origin of ScrollSets? ================================= LLM dataset generation is a _major_ breakthrough in datasets. ScrollSets are, at best, a minor improvement. They are designed to work alongside LLMs to help solve the Dataset Needed problem. https://breckyunits.com/dataset-needed.html Dataset Needed ScrollSets evolved out of TrueBase. ScrollSets have eliminated the need for the TrueBase software (and existing TrueBase sites should be migrated to ScrollSets), but were informed by the TrueBase build experience. https://truebase.treenotation.org TrueBase Although ScrollSets are designed for a world with LLMs, the design is meant to be useful without them as well, and would also have been mildly useful 30 years ago. What were the design goals? =========================== - Have an LLM do the bulk of the work while humans supervise to remove hallucinations. - Can store everything (documentation, schema, all concepts) in 1 clean plain text file or split into many files (using the `import` parser). - The ScrollSet syntax balances _looseness_ useful in creative thinking with the _tightness_ needed by tabular data visualization and analysis tools. Why are measures and concepts root-level features and not indented? =================================================================== The normal way to implement this in Scroll would be something like: ``` measures id string moons int concept id mars moons 2 concept id jupiter moons 63 ``` The flat design was chosen for ergonomic reasons. ScrollSets seem like they might be useful enough to be worth breaking from Scroll convention a bit. Like all things in Scroll, ScrollSets are an experiment, and maybe this design will evolve. Extended Example: a Planets ScrollSet ===================================== Below is the ScrollSet embedded in this Scroll file. Measurements of the measures ============================ Extended Measures Example ========================= Extended Concepts Example ========================= Related ======= Built with Scroll v166.0.0 |
04/21/2024 |
1148 |
5.7 |
| Introducing Program Links |
Introducing Program Links ========================= by Breck Yunits https://twitter.com/breckyunits Breck Yunits May 8, 2023 May 8, 2023 Some web apps are designed to load _entire programs_ from a link. But if a program contain certain characters, such as _newlines_, those links won't work as-is. To create working links you have to run them through `encodeURIComponent`. * If you are creating blog content by hand this is annoying. To update a program link you have to keep a copy of the program, update that, then run `encodeURIComponent`, then paste the result into your post. Today's release of Scroll includes a small new addition that solves this problem. You can call them "Multline Links" or "Program Links". * Here are two examples ===================== Here is a program link to the Scroll web app. https://try.scroll.pub/#scroll%0A%20%23%20Testing%20Program%20Links%0A%20It%20worked!%0A%20%20style%20color%3Agreen%3B%20font-size%3A%20100px%3B program link If you click that link you should see "It worked!" in big green text. * Now let's show a complex real world example. The link below contains a 28 line program pasted verbatim from the Ohayo data science studio. See code Discovery of the Elements https://ohayo.breckyunits.com?filename=test.ohayo&data=doc.title%20Discovery%20of%20the%20Elements%0Adoc.subtitle%20What%20is%20the%20growth%20in%20known%20elements%20over%20time%3F%0Asamples.periodicTable%0A%20hidden%0A%20fill.missing%20Year%201000%0A%20%20hidden%0A%20%20columns.keep%20Element%20Year%0A%20%20%20hidden%0A%20%20%20rows.sortBy%20Year%0A%20%20%20%20hidden%0A%20%20group.by%20Year%0A%20%20%20hidden%0A%20%20%20rows.sortBy%20Year%0A%20%20%20%20hidden%0A%20%20%20%20rows.runningTotal%20count%0A%20%20%20%20%20hidden%0A%20%20%20%20%20vega.bar%20Number%20of%20Elements%20Found%20Each%20Year%0A%20%20%20%20%20%20xColumn%20Year%0A%20%20%20%20%20%20yColumn%20count%0A%20%20%20%20%20vega.line%20Cumulative%20Number%20of%20Elements%0A%20%20%20%20%20%20xColumn%20Year%0A%20%20%20%20%20%20yColumn%20total%0A%20vega.scatter%20Year%20of%20Discovery%20by%20Atomic%20Number%0A%20%20xColumn%20Year%0A%20%20yColumn%20AtomicNumber%0A%20tables.basic%0A%20%20rowDisplayLimit%20200%0Adoc.categories%20chemistry Discovery of the Elements If you click that link you should see a few data visualizations generated from that program. Edit the source and the link will update accordingly. Scroll handles the `encodeURIComponent`. * Why not use HTML directly? ========================== Modern browsers do their best to make URLs well-formed and you will notice that they auto encode certain characters like spaces. But by the HTML spec newlines are not a URL code point and generally are stripped and collapsed. https://url.spec.whatwg.org/#url-code-points URL code point https://infra.spec.whatwg.org/#strip-and-collapse-ascii-whitespace stripped and collapsed Program Links in Scroll handle encoding newlines and any other special characters so you can focus on your content and not the encoding. Built with Scroll v166.0.0 |
05/08/2023 |
510 |
2.5 |
| Counterpoints: A new tool for thought in the Scroll Language |
Counterpoints: A new tool for thought in the Scroll Language ============================================================ by Breck Yunits https://twitter.com/breckyunits Breck Yunits April 13, 2023 Improve your writing by welcoming counterarguments ================================================== April 13, 2023 There's a tiny new symbol in Scroll today: `!`. https://scroll.pub/ Scroll The exclamation mark stands for *Counterpoint*. "Counters" aim to help writers strengthen their ideas by encouraging the integration of counterarguments throughout their essays. You can write your own counters or invite counters from friends, LLMs or Internet commenters. * Counterpoints are easy to use: just start a line with `!`. For example: ``` Humans will eventually establish colonies on Mars. ! But we haven't colonized the Arctic, 1,000x easier ``` You can go as deep as you want in countering your own counters! ``` Humans will eventually establish colonies on Mars. ! But we haven't colonized the Arctic, 1,000x easier - Colonizing Mars is 10,000x more valuable ``` * The Benefits of Countering ========================== When we have a lot of terms for a thing, it's usually a hint that it's a valuable thing. And we have a lot of terms for countering. To name a few: - "Steel manning" - "red teaming" - "playing devil's advocate" Countering is a helpful habit that helps clarify your thoughts, understand and empathize with other perspectives, fix logical blunders, hone your pitch, and improve at truth-seeking. Counterpoints in Scroll are in a sense just a reminder to frequently use these techniques. * The Downsides of Countering =========================== Addressing every plausible counter in your main thread can be difficult if not impossible. It can make your writing too verbose and lose reader interest. Counters in Scroll give you a place to record every scattered counter as it comes to you. You can then choose which are important enough to address in your main argument and relegate the rest to the source code (for the very avid reader). * Implementation ============== Counterpoints do not currently appear in the compiled HTML. Instead you only see them during write time. For now, they are just an alternate syntax for comments. But though they perform no new function, like traffic lines, sometimes form is function. That being said, there are potentially interesting directions this could go in the future, and it would be interesting to hear ideas of what people would like in a v2. * Keep writing, keep thinking! ⁂ Built with Scroll v166.0.0 |
04/13/2023 |
391 |
2 |
| Introducing the Blink Tag |
Introducing the Blink Tag ========================= by Breck Yunits https://github.com/breck7 Breck Yunits October 2, 2022 October 2, 2022 *Scroll's new blink tag lets you call attention to something important* I am really happy with the current state of Scroll and so grateful for the many people who have helped us get it to this point. That being said, I had a feeling we were missing something important. * *So I invented something simple and completely novel* Introducing the `blink` tag. When you have something important to say and you want to interupt people's lives, use the `blink` tag. It's as easy as this: ``` blink This blinks ``` Advertisers have known for ages that human eyes evolved to respond fast to motion. Now, using my new invention, the `blink` tag, you can bring this innovation from advertising to your products and websites. *The blink tag, available now in Scroll 34.1.0* https://scroll.pub/ Scroll 34.1.0 _Note: please don't use this. It's a joke_ Built with Scroll v166.0.0 |
10/02/2022 |
176 |
0.9 |
| The Power of Indented Heredocs: Markdown, Textile, and BBCode in one file with no escaping |
The Power of Indented Heredocs: Markdown, Textile, and BBCode in one file with no escaping ========================================================================================== by Breck Yunits https://github.com/breck7 Breck Yunits August 23, 2021 Scroll is a new language and static site generator that is mostly written in Parsers which are both built on a new syntax called Particles. * In this demo, I extend Scroll by adding support for Markdown, Textile, and BBCode. I want to demonstrate how easy it is to compose many languages into one using Particles, which is due to what you might call the Indented or Off-side Heredoc pattern. https://en.wikipedia.org/wiki/Markdown Markdown https://en.wikipedia.org/wiki/Textile_(markup_language) Textile https://en.wikipedia.org/wiki/BBCode BBCode My implementation is still _researchy_—Parsers still needs a lot of work—but my main point here is to demo how _simple and powerful indented heredocs are_. They allow you to compose unlimited languages in one file in a clean and scalable way. You don't need escaping. I think this is a very helpful pattern you can use in your own languages and code. Traditional Heredocs ==================== Traditional heredocs use start and end delimiters, like these examples from Ruby and Python and Markdown: ``` # Heredoc in Ruby puts Particles is that it is just indented or off-side HereDocs. Or in colloquial academic speak _The Off-Side Rule is all you need_. Most people learn of the off-side rule via Python. Imagine if your whole language was just the off-side rule. That's basically Particles. But my code is not the important thing—the important thing is this pattern. Try it yourself—I bet you will love it. https://en.wikipedia.org/wiki/Off-side_rule Off-Side Rule https://en.wikipedia.org/wiki/Here_document HereDocs Extended Examples ================= Markdown ======== Textile ======= BBCode ====== Built with Scroll v166.0.0 |
08/23/2021 |
306 |
1.5 |