Automating All the Things.

Cutting the Cord: How We Did It.

Ok, so we’re going to take a break from our usual geekery to talk about watching TV without having a traditional Cable TV subscription. There’s this recurring discussion that keeps happening on our town Facebook group and I keep ending up going through parts of what I’m writing here.

Verizon FiOS Bill — A Sample FiOS Bill From Someone in My Town

My mind was absolutely blown this morning when I saw a picture of one person’s FiOS bill. It’s shown at the right. So that’s almost $2600/year.

Many folks will suggest you just go with Internet-only packages and a streaming service like YouTube TV, Hulu Live TV, Sling, or whatever, with Smart TVs or set-top devices like Apple TV, Roku, or Fire TV, and it’s tough to argue with the simplicity of that solution. It’s pretty much a turnkey and go type of solution. Plug it all in, turn it on, login and you’re watching TV in a few minutes, even local channels with sports. Checking out our neighbor’s use case here and substituting in what many in town are paying, she could move to an Internet-only plan for about $75 a month, and use one of the many streaming services for about $55-65/month, depending on who you picked, and you just turned $2600 a year into $1700 a year. What could you do with an extra $900 in your budget? Only thing though, we made some assumptions here, like equipment – we’re assuming that you’ve got all Smart TVs that have the ability to accommodate whatever streaming service(s) you’re planning to use. Otherwise, you’re buying a device to hook up to the TV. That’s going involve some up-front cost, which will eat into that $900, at least for the first year.

The solution I’ve built out at our house involves more up-front expense, but involves no recurring costs apart from our normal Internet access costs. In our case, this is the Verizon FiOS Gigabit plan. We’ve had FiOS Internet since we moved into our current house, about 15 years ago, back when it was 25/5 service. How the times have changed!

We’ve got a roof-mounted antenna that we installed a couple of years ago when we made the big switch. Not being the “climbing on the roof” type, I contracted this part out. I found a local company that did the job, including the antenna (a Channel Master CM-3016, now called the Advantage 45). Rather than connecting to a TV, the antenna is connected to a networked TV tuner device. In our case, we’re using an HD HomeRun (an HDHomeRun Connect 4K). This unit has 4 digital/ATSC tuners, 2 of which are ATSC 3.0 capable, and connected to our network using a wired Ethernet cable (not WiFi). ATSC 3.0 is the new standard that’s rolling out that supports 4K over-the-air broadcast TV.

So, that’s got local TV signal from the airwaves into the house and onto our network! How do we watch it? There are a couple of ways. SiliconDust (the folks who make the HDHomeRun box) offer apps for the major streaming devices and Smart TVs that let you access the device, decide what channel to tune, watch TV, even pause live TV. What’s not there though is all the DVR capabilities you’d want to have. No ability to record shows. So let’s talk about how I’m getting that.

To get DVR functions. I’m using the Plex Media Server. I’m running Plex on a custom-built server on our network that runs the Ubuntu Linux operating system, with our media files stored on a Synology Network Attached Storage array. We’re using the Network File System (or NFS, if you’re a Unix type) to mount the drive on the Plex system from the NAS. It works really well. Plex has great support for the HD HomeRun devices too. You go into the Plex settings, tell it to take a look around your network, it finds the tuner, scans for available channels, you pick the ones you want to make available, then Plex goes and grabs guide data, just like you’re probably used to having on a cable box.

Once you’ve reached this point, you’ve got access to a pretty normal looking programming guide, and even have familiar looking DVR features like recording single episodes, or all upcoming episodes. If you decide to do the latter, you’ve got even more options at your disposal, like the ability to record only new episodes, or choose how many episodes to retain, how long to retain episodes, if you prefer HD episodes, or only want HD episodes. Maybe you only want to record from a specific channel, or only at a certain time. Those options are useful when it’s a show that’s syndicated and on multiple channels at multiple times. Then there’s extra fun stuff like built-in channel skip. You can mark for channel skip or let Plex take the wheel and try its hand at just chopping the commercials out totally on its own. I’ve had a mix of experiences both good and bad letting it go on its own. Sometimes it’s great, and other times, I’ve missed Final Jeopardy during a tight match. So, I don’t typically go beyond “Mark for Skip”. Honestly, it’s pretty accurate.

So, now that I’ve described how the bits fit together, let’s actually take a look at a diagram (Click to enlarge). It’s more of a logical diagram, really. I work in networking, so I prefer to keep network elements like my routers, switches and WiFi Access Points separated, so those are all individual pieces in our house. In yours, they’re probably not. You may be used to a converged device that’s a Router/Switch/and WiFi Access Point all-rolled-into-one, and that’s alright. You do you! But, this is how I’ve built what we’ve got. The different colors indicate different types of connections. Black is the coax from the antenna to the HD HomeRun. Red is the fiber optic cable from the street the Verizon Optical Network Terminal (aka ONT) on the side of the house (in real life, it’s probably yellow, but yellow on a white background? really hard to see!). Blue is plain old Ethernet cable.

So, if you were going to build something like this, what would you want to buy, what would it cost, and when would you start saving money compared to getting taken to the cleaners by Verizon? Great questions. The answers really depend on what you like to watch for TV. If all you watch are local network shows, you’ll recoup your investment in about a year. If you watch other stuff, it might take longer. If your mobile phone plan includes streaming services for free, as many do, that’s a bonus. For example, we’re a T-Mobile house. We get free Netflix thrown in with our family plan. Some Verizon customers get Disney+ for free. There are tons of deals from carriers, figure out what works best for you and exploit it to your advantage.

If you want a simpler more “turnkey” type experience, I’d suggest you look toward the Synology NAS as a solution. Plex runs on Synology, works great, and your storage is built right in there. There are trade-offs, but honestly, if you’re savvy enough to know that you’re impacted by those trade-offs, you’ll also know if the value proposition is there for you to spend more to work around it. You could shave some $$ by building your own small server out of something like a BeeLink Mini PC, available from Amazon, install something like Ubuntu Linux on there and run Plex on it yourself, but there’s a much steeper learning curve involved in that path.

If you’re interested in pursuing this idea, check out the diagram at the right for the view of the simplified version of what we’re running at our house. You’ll note the use of the Verizon-provided FiOS router, and the Synology NAS, or the small server. But the rest is pretty much the same. Now of course, you don’t have to use that same Channel Master antenna I did, but I’m here to tell you that it’s pretty difficult to beat their performance, especially at their price point. Speaking of price points… On to what you’ve all been waiting for. The numbers. First, some assumptions I made in my analysis… I looked at my neighbor’s use case up above. 7 TVs, with a DVR requirement. I kicked the Internet speed up from 200M to Gigabit using what I’m paying per month on the autopay discount to do the cost modeling. The full comparison is a 3-Year Total Cost of Ownership (TCO) Model. First up, building with the Synology NAS, with redundant 6TB of redundant storage, then the Small Server build, followed by keeping the FiOS Triple Play Bundle at $214.99 a month. That’s the easiest one to calculate, of course, since it’s just 36 months of service at $214.99.

Item	Link	Unit Price	Qty	Total
NAS	https://smile.amazon.com/dp/B087Z6SNC1	$399.99	1	$399.99
Hard Drives	https://smile.amazon.com/dp/B085Z4P89R	$129.99	2	$259.99
Plex Software	https://www.plex.tv/plex-pass/	$119.99	1	$119.99
Antenna w/Install	NJ-based Installer Company I used	$250.00	1	$250.00
HD HomeRun	https://smile.amazon.com/dp/B092GCN9NL	$199.99	1	$199.99
Roku Stick 4K+	https://smile.amazon.com/dp/B09BKCQYRN	$69.00	7	$483.00
FiOS Gigabit		$75.00	12	$900.00
				$2612.95

Converged Plex, Synology Build, Year 1 Total Cost of Ownership

Item	Link	Unit Price	Qty	Total
BeeLink U59	https://smile.amazon.com/dp/B09H5QXS6K	$250.00	1	$250.00
2TB SSD	https://smile.amazon.com/dp/B089C6LZ42	$187.66	1	$187.66
Plex Software	https://www.plex.tv/plex-pass/	$119.99	1	$119.99
Antenna w/Install	NJ-based Installer Company I used	$250.00	1	$250.00
HD HomeRun	https://smile.amazon.com/dp/B092GCN9NL	$199.99	1	$199.99
Roku Stick 4K+	https://smile.amazon.com/dp/B09BKCQYRN	$69.00	7	$483.00
FiOS Gigabit		$75.00	12	$900.00
				$2,391.74

Converged Plex, Small Server Build, Year 1 Total Cost of Ownership

As you can see in the graph shown at the right, the Small Server and the Synology NAS builds are quite close in the 3-Year TCO model, landing about $200 apart across the entire time. Given this, unless you’ve got serious reasons to pursue the Small Server route, I’d recommend going down the Synology NAS route. You get built-in drive redundancy, more storage space, and a turnkey web-driven management interface that falls out of the box ready to go.

Please, don’t forget, there’s also nothing wrong with just simply dropping the Triple-Play and picking up a streaming TV service. You’re going to cut your monthly expenses in half here. But, if you want to go deeper, and learn a few things along the way, you can. This is the way.

The Robots are Speaking…

So many applications and processes involve notifications. As we begin to automate processes, it becomes increasingly important to know if tasks we’ve initiated succeeded or not. That’s where automated notifications come into play. In my case, often times, these take the form of chatbots that send messages to let me know that something happened, or that a process either succeeded or failed.

I’ve settled on Telegram as my platform of choice for this. What’s attractive about Telegram? It’s available for a wide variety of platforms – iOS/iPadOS, Android, macOS, Linux, Windows, there’s even a web client.

So, how does one create a bot? You have a conversation with the BotFather. BotFather is a Telegram bot that creates bots (how meta, right?) for you. The bot walks you through the whole process. When the process is complete, you’re given a token – this is a secret value that serves as the “keys” to the bot. Once you’re done this process, you’re almost ready to start using the bot in your automations. Next, you start having a chat with the bot, just like you would with another human. Send the /start command to the bot to start it up. Once you’ve done that, you want to figure out the chat_id for that chat. This is pretty straight forward to do, fortunately.

To get the chat_id value for a private chat with your bot, you visit this URL:

https://api.telegram.org/bot[your-bot-token]/getUpdates

Want to use the bot to notify multiple people? Create a Group Chat and add the bot to it. Send a message to the group and reload that URL up above. Check the updates for the Group Chat messages, and look for the chat_id for the Group Chat in there. How can you tell the difference between a regular chat with the bot vs a Group Chat that the bot belongs to? Easy. All chat_id values are 32-bit integers, but Group Chats ID values are negative, while individual chats are positive. Easy, right?

Ok, so you’ve got the Token, and a chat_id value. It’s time to put those bits to work. Here’s an example, in Python, using the python-telegram-bot module, which you can install using pip install python-telegram-bot.

#!/usr/bin/env python3

import telegram

myToken = '123456789:ABC-123456-foobarbaz9876543210'
chatID = -123456789

bot = telegram.Bot(token=myToken)
bot.sendMessage(chat_id=chatID, text="Hello World!")

This is just one of many ways possible to send notifications. Find the one that works best for you and your workflows and go with it! If you want to take a shot at using Telegram, give their docs a read before you start.

Juniper Switch Port Bounce

How many times do you want to bounce a switchport? Ok, it’s not every 5 minutes, I’ll grant you that. But when you need to, you need to. There’s a handful of strategies we can employ to do this.

Firstly, wild-west style. Just walk right up, yank the cable out, count to 10, and shove it back in. Did it work? Did I grab the right cable? Shoot, I hope so. Wait, Juniper starts counting at zero and Cisco starts counting at 1. Oh crap. I pulled the wrong cable. Let’s go back and do it again. Once more, with feeling, and the right cable this time.

Or, we could take the vastly more measured approach of writing up a full MOP, taking it to the Change Control team, getting it approved, scheduling a change window, coordinating with testing teams, double-checking that we’ve got the right cable, then pull it out, count to 10, plug it back in, have the testers verify that everything works correctly, close out the change window, and then go to bed. But that seems slightly excessive, especially if we really need to bounce that port right now, since the thing on the other end’s not responding and we’re troubleshooting because there’s no connectivity.

What if we take the middle-ground? What if we automated the process a bit to lower the risk of some of the human error factors? If we know what port we want to bounce, we can make that happen in a measured, programmatic way through the Junos Python API, which of course, uses NETCONF under the hood.

Enter the Python script I wrote last night. It’s written (naturally) in Python 3, since Python 2 is now EOL, as of a couple of years ago. Seriously gang, if you’re still writing in Python 2, stop. Anyhow, I’m on the road for a couple of days for work, and after a drive last night, and some time stuck in traffic, and some dinner with a work contact, I was just relaxing, and I wrote this.

Yeah, I know, weird way to relax, right? Ok, I had been pondering this the other day, and just sort of threw the idea in the background for processing at a low priority. You know how that goes. Wrote a bit of code, cranked up the VPN back to home, experimented with bouncing the link connected to a Raspberry Pi on the network at home a few times and here we are.

Feed the script a hostname/IP for the switch, (optionally) a username – if you don’t, it will default to whatever your environment resolves for $USER, (optionally) a password – if you don’t, it will expect to be trying to authenticate using SSH keys, and the port you’re looking to shut and turn back up. Using the Junos Python API, the script connects, does an exclusive config lock, disables the port, commits the config, rolls back, commits again, and finally unlocks the config.

At any rate, here it is, in all its splendor… I also copied and pasted most of the same code and at the same time wrote a “PoE Sledgehammer“. It disables PoE on the switch, then rolls back the change. Useful if you need to do something like simultaneously reboot every phone and/or WLAN AP connected to the switch at the same time. As the name implies, it’s kind of a blunt instrument. Use it with caution…

Smartening Up the Washer

I solved this problem once a couple of years ago using a Wemo smart plug, IFTTT and WhatsApp. Well, fast forward a couple of years, and everything broke. Wemo went and totally broke their IFTTT integration, IFTTT completely changed their model pricing model, and Facebook really changed how they were handling how involved they were (and what level of privacy they were giving to) with WhatsApp. So, given how broken things were, I had to go back to the drawing board.

After a conversation with a guy from work that does a bunch of projects like these, I settled on one of the Etekcity smart plugs from Amazon that uses the VeSync app. These days, these seem harder to come by. If I was going to do this over again, I’d probably do it with a Kasa smart plug and use their local API. Anyhow, the VeSync app also has an unofficial, but pretty well-defined, and stable HTTP API that works really well.

So we’re going to leave IFTTT out of the party this time around, just read directly from the plug’s API, figure out how much power is being used to determine whether or not the washer is running, and once we know the washer has stopped, we alert the family by way of a group text. While in the past this was a group text via WhatsApp, now it’s a message to a Telegram group using a bot. I’ve got no great affinity for Telegram beyond the fact that it’s easy enough to setup the bots and get everyone setup on it.

The bottom line? All of this together enables a pure-Python solution that runs under current Python 3 releases, plays super nicely inside a Docker container, which is how I choose to run it. In essence, the code is pretty simple – turn on the plug, start up an infinite loop where you keep reading power levels. After you change to “ON” state, wait for power to drop below a line, to go back to “OFF” state, at which time you throw out a Telegram message to notify the family that the load in the washer finished up and it’s time to go downstairs and move the wash to the dryer.

This has really been super effective at reminding us to stay on top of the laundry. The number of times that we forget loads in the wash and end up needing to re-wash because stuff got forgotten and ended up getting funky smelling has been slashed down to nothing. Great stuff!

Grab the code, or deploy a container today!

It Started With A Light Bulb…

One night a few years ago my wife and I sat in the living room watching something on TV, when suddenly one of the recessed lights went out. The bulb died. It wouldn’t be long before a great adventure would begin.

The next morning I trotted off to Lowes to pick up a replacement bulb. I decided that it was time to catch up with current tech and move from the power-gulping bulbs we had in the fixtures to newer LED replacements, so I picked up 4 of those retrofit kits. They’re simple to install. You pop off the old trim ring and unscrew the old bulb, screw in an adapter, connect the wires from the adapter to the LED/trim piece, and put LED/trim into position. Installation takes a good 30 seconds. Minutes later, the 4 cans in our living room had been completely modernized. After I finished and went back to the dimmer on the wall, I popped the little “slider” piece back in to re-energize the switch and the LEDs started flickering. Uh oh. So, the old Lutron dimmer in the box wasn’t ready for LEDs. It’s minimum load was too high, and so it was passing enough power that it was lighting up the LEDs.

Unwilling to go back at this point, I returned to Lowes in search of an updated dimmer. It was then that I was greeted by the Lutron Caseta family of products. On sale was the starter pack. For a small premium beyond the cost of the dimmer I was already going to buy I could get “Smart” switches that I could control from my phone with an app, and even worked with Apple’s HomeKit. I was sold already. Within a year most of the switches in our house had been converted to either Caseta dimmers or switches, except for the couple of spots where we’re using Hue lights.

Is there more? Oh yeah, there’s more. Wait until I tell you guys the story about the microcontroller, fire, the sensor from China, Python, and the Raspberry Pi.