Got it!!!
I had the replacement dipstick tube on hand so I knew how much was stuck in the engine. The broken dipstick in this video is much shorter but having the replacement on hand let me know I'd have to do something slightly different. But that video did mention it's probably not a good idea to drop something into the oil pan. So.
Step 1. Get a Paper Mate #2 pencil and some earplugs. Take the spring out of the pencil and screw it into the earplug. Squish the ear plug and shove it deep enough into the dipstick tube that it will be out of the way. Once the earplug expands unscrew the spring and set it aside. Now you don't have to worry nearly as much about something falling into the oil pan.

Step 2. Shove a plastic screw anchor into the dipstick tube. I was super duper lucky because I had one exactly the right size. Don't force it all the way down if it doesn't want to go that far. Once that's in screw in the screw.

Step 3. Initially I grabbed pliers right after step 2 and tried pulling the tube out. I couldn't get the right pulling situation setup though and had to start over a couple times. What eventually worked was to tie some straps in a knot around the screw, so that they catch on the screws head, and use the straps to pull the tube out.

Step 4. For me the tube would rotate but really did not want to be pulled out. I had the earplug in there so that gave me some peace of mind about pulling hard on the straps. It only came out after pulling very hard.

Sweet, sweet, sweet, sweet Victory.