Frog Twaddle

I will not give up my power. I read this on the small web today and it helps. Maybe someone else will find this useful.

I Am Part of a Whole

As I was cleaning up some of my older files today and moving them off device, I wondered if I could use AI to build a manifest that I could keep on device in case I ever needed to find one of the folder files again. As someone who generates a lot of files, it's important I have good drive hygiene and I'm always looking to up my game.

Getting a list of the files into a document is easy. You can simply list a directory and redirect it into a file. On Mac this can be done with something like,

ls -lh >> myFiles.txt

It's very basic but it does create a simple file list that can be easily searched. Sometimes, with me anyway, the file name is not enough for me to recall what information a document contains. This is where I thought that bringing AI into the mix may help up my game.

After some minor fiddling, I came up with the following,

#!/bin/zsh

# Define the output file
output_file="output.md"

# Clear the output file if it exists
> "$output_file"

# Find all Markdown files recursively and process them one by one
find . -type f -name "*.md" -print0 | while IFS= read -r -d '' file; do

  # Run the temp command with the content of the current file
  result=$(cat "$file" | ollama run llama3 "Briefly summarize the text in this file. The summary should be no more than 3 sentences in length. Suppress any response that is not part of the summary. If the file is empty just respond with the phrase 'Empty File'")
  
  # Append the formatted output to the output file
  echo "---\n" >> "$output_file"
  echo "**${file}**" >> "$output_file"
  echo "\n" >> "$output_file"
  echo "$result" >> "$output_file"
done

In this case, the AI I am using is Ollama with the llama3.1:8b model. In my experimenting, this model provided decent summaries with the correct prompt. As you can see in the code block above, I give the model the following prompt.

“Briefly summarize the text in this file. The summary should be no more than 3 sentences in length. Suppress any response that is not part of the summary. If the file is empty just respond with the phrase 'Empty File'”

I've found that the llama3 model can be a bit chatty so I give it multiple guardrails to keep it on task. One: be brief. Two: limit the response to 3 sentences. Three: don't provide anything other than a summary. Four: if the file is empty, just say “Empty File” and move on. I will say that even with those guardrails, Ollama will still occasionally give some commentary, but it's knocked down enough that the output is useful and only requires some minor cleanup. In a future revision, I'll probably refine the prompt as well as provide some hinting so it has a format to follow.

As I see it now, this approach could be easily adapted to solve multiple problems and could be adapted to more sophisticated programming languages like Python to do even more. For now, though, I've scratched my itch and it's time to archive some files.

I was recently asked this question by my wife's dad. I decided it might be fund to do a little back of the napkin style math. Below are my calculations.

To solve this problem, we need figure out what a “standard” printed page consumes in storage. According to Microsoft, an average typed page on US Letter-sized paper (8.5 x 11 inches) typically contains about 2,500 to 3,000 characters, including spaces. This estimate can vary based on factors such as font size, typeface, and spacing. For example, using 12 pt Times New Roman with standard margins, you might expect around 2,500 characters per page. For our estimate, we’ll use 3,000 characters to be conservative and account for smaller font sizes.

Next, we need to determine how much storage each individual character requires. MacOS (my platform of choice) and other modern operating systems support a wide variety of encodings, but the primary encoding on MacOS is UTF-8. A regular text character is 8 bits (1 byte), but to support other characters like emoji, Kanji, etc., we need twice that. Therefore, UTF-8 uses 16 bits (2 bytes) per character for these cases.

Now we can calculate the total storage capacity. Five gigabytes is equivalent to 5 billion bytes or 40 billion bits. Since an average page contains 3,000 characters (or 48,000 bits), 5 gigabytes of storage can hold over 800 million (833,333,333) US Letter pages of text.

For fun, according to the blog Reedsy, the average number of characters in an English-language novel can vary significantly based on factors such as genre and writing style. However, a rough estimate can be made using the average word count of a novel, which typically ranges from 70,000 to 100,000 words. Assuming an average of 5 to 6 characters per word (including spaces and punctuation), we can estimate the total character count as follows:

For 100,000 words: 100,000 words × 5.5 characters/word ≈ 550,000 characters or ~9 million bits. This means our 5 gigabytes of storage could hold roughly 4,444,444 novels.

Is the above useful? No. Accurate? Meh. Fun to consider? I thought so!

Kris and I cleared Alberta today and tipped into Saskatchewan which, in my opinion, is the most fun Canadian Provence to say out loud. Getting to our final spot for the night though required that we cover some serious ground. As of this evening, we're more than 2,000 km into our trip and we saw some pretty neat things along the way. If you were to look at a map of the North American continent, you'd see that we're the equivalent of only about ⅔ across Montana. This is somewhat surprising as the landscape is more like what you'd see in South Dakota except much more green. The wind is similar to what you'd expect in the prairie as well and we had make way against a headwind that I'm sure didn't help our gas milage. Thankfully, the 4Runner is more than up for the task and covered the undulating hills with little difficulty.

One of the highlights of the day was an accidental discovery of the Battelford Tail Ruts which just happened to be near a gas station and a field where we could let Denali stretch his legs. According to the plaque,

From 1882 until 1891 Swift Current was the closest railway shipping point for the populous Battleford district. Trade, especially the buffalo bone trade, created a heavy volume of ox-cart traffic between the two centres while Battleford was under siege during the Riel Rebellion, Swift Current was strategically important to the entire country.

And so, you can see the actual ruts from the Ox Carts more than 100 years ago today!

The other nice thing about this site... a geocache. This was my first Canadian Geocache which earned me the Canada AND Saskatchewan souvenirs. Sweet!

We also passed the time listening to a documentary-style podcast called Wilder which talks about the Little House on the Prairie books, Laura Ingall's Wilder, her relationship with her daughter, and the legacy of those books and their lives. It was quite appropriate given the day's scenery.

Finally, we ended our day at a cozy camp site called Bin There. Kris and I were both happy to not have to set up the trailer for an evening and instead get hot showers, wash laundry and setting into a real bed for the evening. In fact, I am writing this as a run our laundry through the wash! Below is a quick shot of our digs for the evening.

And with that... I'll call it an evening! Stay turned for the next installment!

As I write this, Kris, Denali, and I are settled in for our third night of camping. Of course, my intent was to blog every day but a full day’s drive followed by setting up camp in the rain (all three nights so far), and wrangling dog have made an evening of blogging slightly ambitious. I’m happy to report, however, that our trip across the Trans Canada Highway has so far been fantastic. We’re more than 1,000 km into our trip and we’re passed the Canadian Rockies. As someone who spent a year of his life in Colorado, the Canadian Rocky Mountains have a grandeur and magnitude all their own. As we wound our way through Roger’s Pass we were treated with endless views of rivers, steep cliffs, and signs warning of mountain goats and avalanches. Alas, I could not take pictures of these things as I was driving, but I did grab some shots from the places we stopped. This marker has a little history of the pass’s namesake.

As it happens, it’s also Canada Day!!

I’ll share more details from the trip in the next couple days. For now, these are two last shots from the start of our trip in WA at Oostema Farm.

This first shot is Denali and I walking out for our morning walk before we crossed the border and embarked (see what I did there) on a long drive.

This second shot is from the end of that day’s drive. Life on the road can be hard.

One final thought. Please be forgiving of this post. I typed it out on my iPhone. : )

A Look Back

This past Wednesday, Kris and I took advantage of the Juneteenth holiday to explore some local history. Earlier in the week, Kris mentioned that the End of the Oregon Trail museum was nearby, and given our imminent departure back to the East Coast, I couldn't think of a better way to wrap up our time here in Oregon.

After a relaxed start, we hit the road around 10 AM and headed for the End of the Trail Interpretive Center in Oregon City. The trip is a short, hour-long drive from our home through rolling terrain that passes fir-lined hills with occasional glimpses of Mt. Hood, punctuating the local beauty. As we made our way to the interpretive center, we saw a sign on Highway 205 for a scenic overlook. It's easy to pass these opportunities up when you live here and are busy with daily to-do lists, but we were purposefully taking a break from our hectic schedule and decided to check it out.

Good call! After we pulled off and parked, we discovered some Oregon history (thematically appropriate) and beautiful views. The photo at the top of this blog entry shows the gem at this stop: the Willamette Falls Locks with Mt. Hood in the background. Several historic markers highlighted the history of Oregon and the Locks, which came into use on January 1st, 1873. There was also a marker celebrating George Abernathy, the first Provisional Governor of The Oregon Country from 1845 to 1849. We encountered the name Abernathy several more times during our visit to the interpretive center. This spot also had a geocache, earning us bonus points for our decision to stop!

The interpretive center is a very short hop from Willamette Falls, and when we arrived, there was plenty of shaded parking. We started our visit by buying our tickets and making a cursory stop at the gift shop. For those who don't know, we enjoy collecting stickers from the interesting places we visit. I could tell this was going to be a fun trip and asked about stickers when we were buying our tickets. No joy. They do carry them, but they were out of stock. This was especially disappointing because the sticker featured the iconic 8-bit image of a wagon pulled by oxen with the classic “You have died of dysentery” line beneath—a favorite for nerd historians everywhere. Oh well. (Don't feel too sad for me; I did get a sweatshirt with the same print instead.)

The good news is the disappointment was short-lived. We started our tour of the interpretive center, where several exhibits highlighted the history of the area. One fun fact: in 1792, American sea captain Robert Gray, in the ship “Columbia,” discovered the mouth of the river that now bears the ship's name. I also learned that Astoria was established by the Pacific Fur Company in 1811. Neat! While there are several exhibits that highlight facts like this, I was more drawn to the personal stories of the women and people of color who worked to make Oregon (now Oregon and Washington) their home. While Oregon was never a slave state, it was apparent from the posted individual histories and other historic documents that this was due to twisted racist ideas and not some higher morality. Reading the stories of these early “minority” pioneers was fascinating and spoke volumes about the human spirit, determination, and kindness. I reflected on where we are today compared to then, and it's easy to see we still have quite a ways to go before we become the best version of ourselves.

After meandering through the static displays and reading several personal histories, we went to the theatre to watch two movies. The first was a traditional museum short that highlighted what it was like to be on the Oregon Trail from four different journal accounts from those early travelers. The second movie was a more recent account of the effects of the Oregon Trail on the native peoples. The latter movie was called Oregon’s First People and was definitely my favorite of the two. It covered how the expansion of the settlers displaced the native population, the immediate and longer-term deleterious effects of the expansion on their culture and population, and the recent efforts to restore the native peoples' ability to live as they had before the expansion. One of the native interviewees shared that he can now go to the mall and hear young people speak in their native language, something that was not likely to happen when he was growing up. The thought that their language is coming back gave me warm fuzzies, and I hope it continues.

We finished the exhibition portion of our visit with a trip through another set of static displays. There was a fun board that quizzed you on your Oregon knowledge with things like: highest point, Mt. Hood; state flower, Oregon Grape; and state rock, thunder egg. The final exhibit leads to a walking path that takes you to the main attraction.

The marker for the End of the Oregon Trail is in the middle of a heritage garden and has two plaques explaining the significance of the site. The marker is at the western terminus of the trail, some 2,000 miles from where the trail begins in Independence, MO. We stood and took the requisite selfies after a brief moment of reading the information. And with that, we had, ourselves, reached the end of the Oregon Trail.

Bonus Content

After reaching the end of the Oregon Trail, Kris and I rewarded ourselves with a trip to the local McMenamins, the Oregon City Pub. Kris gets full credit here as she had seen this place many times while running errands in the area. The pub was originally St. Paul's Episcopal Church, which dates back to 1930 with an associated history reaching back to the 1860s. It sits next to another historic place, the Oregon City Bridge, which was put into service in 1922 and is registered with the National Register of Historic Places. After a meal of Fish & Chips for me and Chicken Strips 'n Tots for Kris, we headed home via the historic bridge.

It was a fantastic day out, and I'm looking forward to more adventures with Kris. Until next time... peace.

I'm a technologist who enjoys discovering effective ways to tackle new challenges. This blog serves as an outlet for me to share what I've learned with you, intrepid reader. Read on to explore empirical management techniques, data capture and retrieval, AI for managers, and much more. I also do other stuff and write about that too.