Our BlogTips, Tricks, and Thoughts from Cerebral Gardens

Solved: My Time Machine and Catalina Issues

macOS Catalina


In a hurry, read the TLDR version below...

Quick summary of the problems I've been experiencing, in case you hadn't seen me ranting on Twitter...

1) Since upgrading to Catalina, I've been unable to complete a single Time Machine backup. Time Machine would continually try to backup, but after days or even weeks, it would eventually fail and I'd have to start it over.

At first I was using an existing Time Machine backup from Mojave. After that failed a few times, I decided to wipe it and start from scratch. Even after starting from scratch, the Time Machine backup would still take forever and then eventually fail. I joked on Twitter... that I was going to post screenshot updates each day showing the progress, but that it would end up being a 6+ month project.

Instead, I filed a FuBAr1 (FeedBack Assistant/radar) and sent Apple some sysdiagnostic and tmdiagnostic files.

2) The second problem I was complaining about was due to my continuously filling SSD that wouldn't give up space no matter how much I deleted from it.

Of course, I was blaming APFS and Time Machine for this, since it's well known now that APFS snapshots are used to keep data available until Time Machine can back it up. I've been using tmutils to delete these snapshots when I needed to free up space, but even after deleting the snapshots, I wasn't getting my free space back2.

I received feedback from my report I'd sent to Apple (Yay, they do work!). They pointed out that the issue was due to a third party app I have installed called Disk Drill by CleverFiles (which I have as part of my Setapp subscription). The app has a "feature" called Guaranteed Recovery that is supposed to help recover files later if you accidentally delete them. It "works" by creating thousands of hard links to what appears to be every file on your system in a hidden directory (/System/Volumes/Data/.cleverfiles/). This means when you delete a file, it's not really deleted because there's a hard link effectively creating a duplicate in the hidden folder. Personally I feel this is a terrible feature since you should have proper backups of your data anyway, so when you delete a file, it should be deleted. (Note: I originally installed Disk Drill in order to attempt to recover images from an SD Card that died, and so had no idea it was doing anything wonky to my main drives).

Over the last month or so, I've been trying to free up space on my main SSD because the OS is constantly complaining that my drive is full. Because I kept getting alerts that I needed more free space, I kept moving/deleting files. Eventually I'd cleared/off-loaded over 500G of data and was still scraping by with about 30G of free space. Now that I'm aware of the issue, I've taken a look into the .cleverfiles hidden folder on my iMac and I see it has over 450G in it. Wow.

First thing, add /System/Volumes/Data/.cleverfiles/ to the Time Machine exception list. This should fix the first issue with Time Machine taking forever to backup. Though, if you are using an existing Time Machine backup, it can still take a long long time for the next backup. My MBP has been working on it's first backup since I added the exception for 2 days still, and it's completed 150G of 177G. I created a brand new backup on my iMac and it completed it's first 650G backup in a day, so I'd recommend just wiping your old Time Machine backup and starting fresh (unless there's something you really need in the old backup, but then, why not restore it first, then wipe etc). Wait until after you read the rest of this post before staring a new backup though.

Next, I went into Disk Drill to disable to the Guaranteed Recovery feature. I don't need it, and think it's a flawed idea. While looking at how to disable it, I see it has a setting for how big the "Guaranteed Recovery Storage" should be. It's set to 8GB by default. Recall my .cleverfiles stats it has >450G. A lot more than 8G. That limit is supposed to be the limit of files hard linked but deleted. But it appears there's a bug in the app because it hasn't been trimming down to the 8G limit (perhaps there's a Catalina issue Disk Drill is hitting that prevents the trimming, and that's why this only started getting really out of hand after I upgraded). I didn't bother digging into this since I'm disabling the feature anyway. After disabling the feature, and "resetting" the storage, I expected the .cleverfiles folder to be removed or at least cleared. It was neither. Perhaps another issue due to Catalina. (Note that Disk Drill does have Full Disk Access permissions on my machine).

I went back into Disk Drill, and it had re-enabled Guaranteed Recovery on each of the drives I had just disabled it on. Another bug? Not sure, no time to dig into it, so instead, I uninstalled the app from my machines. Note that you can't just delete the app, you need to follow the full uninstall instructions here (modified slightly if you have the Setapp version): https://www.cleverfiles.com/help/how-to-uninstall-disk-drill/

I then manually deleted the /System/Volumes/Data/.cleverfiles/ folder.

One issue is that I can't remove a folder: /Library/StagedExtensions/Library/Application Support/CleverFiles due to it being read-only in Catalina. I supposed I could turn off SIP and delete it then, but I try not to do things normal users won't/can't do in order to keep my system consistent for testing.

After deleting .cleverfiles, finishing a full Time Machine backup, and ensuring there are no remaining APFS snapshots, I now have 593G free! Subsequent Time Machine backups are completing quickly again, and I expect my whole machine will start behaving better again (time will tell).

My thanks go out to Apple staff who responded to my report and helped restore (some of) my sanity! Hopefully the developers at CleverFiles can fix the issues they have in Disk Drill.

TLDR: It wasn't Apple's fault. A third party app, Disk Drill by CleverFiles was causing the issue. It was also causing the issue that prevented hard drive space freeing up as I deleted files. The apps not so clever feature was creating hidden hard links to basically every file on the system so you could recover it later in the odd chance you accidentally deleted a file.↩︎

Footnotes:

1: I'm still trying to make this nickname for the new Feedback Assistant reports stick, but I'm probably the only one. 🤪↩︎

2: I had used Daisy Disk to try and locate the missing disk space, but it didn't help find them. It did have a huge chunk (4-500G) marked as hidden, but didn't break down the location.↩︎

Why did I reserve a Tesla Cybertruck?

Tesla Cybertruck


Abbey Jackson messaged me on Twitter:

I started to reply with a tweet, then figured it would take a few tweets so started a thread, then realized this needed to be a full blog post...

First, note I'm not a truck guy. I'm an SUV guy. But I appreciate how useful a truck can be when needed (I've borrowed my bro-in-law's F150 on more than one occasion). The Cybertruck looks like it will feel more like an SUV than a truck, and actually, I could see it shipping with an SUV option at some point.

Stainless steel is awesome. I've always wanted a DeLorean even thought about electrifying one as a side project at some point. The strong panels are a major bonus. My Bolt was dented when ice slid off my house and hit the hood during last spring's thaw. Over $1000 in damage. Also, I'm paranoid about a-holes keying Teslas. Granted there's a low probability of it happening, but I figure if it'll happen to anyone, it'll happen to me (I'm just that lucky).

The transparent steel windows look great (regardless of them failing during the demo). Making it harder for thieves to smash and grab will be a benefit since Telsas are often targets. Impressive to see a company actually use transparent metal too, it shows they're reconsidering everything, even if it's already "a solved problem".

Then of course, there's the non-tech aspects. I'll admit I've always liked the Hummer. But their gas guzzling nature of course has always prevented me from really looking at getting one. It might just be me, but drivers of Hummers and similar vehicles always seem to give the impression they just don't care about the world, and almost take pleasure in actively hurting it. Almost like they're shouting: "Look at me, obnoxious as hell, f*king up the planet!". Driving the Cybertruck feels like it'll be similar to me shouting: "Look at me, obnoxious as you, but helping the planet!"

I'm always trying to convince others to go electric when purchasing their next new vehicle. But everyone always has an excuse about why they have to stay with gas, and driving the Cybertruck will be proof that most of their "reasonable" excuses just aren't valid anymore (and I'll be able to take them for a ride to demo why):

"I drive to Florida and back once a year so I need 2400 km range (cause I drive for 24 hours without stopping to eat or pee)"
  • — The tri-motor version will have 800 km range! That means you can drive to Florida (from Ontario) only needing to stop twice at a supercharger, that's less than the number of times you'd have to stop for gas.

"I have to use my vehicle for work, so it needs to be rugged, tough and able to carry all my stuff"
  • — Payload 3500 lbs! Tows 14,000 lbs! My Cybertruck will crush your puny work truck. 'nuf said.

"Electric vehicles cost too much (because I suck at math and can't add up the deferred costs of gas, maintenance, or factor in the environment damage we'll have to pay for later)"
  • — $40-70K USD! With nearly zero maintenance costs, really low fuel costs, probably cheaper insurance costs, and no damage to the climate to pay for later1.

"I'm special and shouldn't have to consider the damage I do to the world, someone else can clean up after me, gas is just easier because I already understand it"
  • — Ok Boomer. There's no convincing these people with logic and facts, I'm trying not to waste too much time and energy on them anymore.

"Electric vehicles are bad for the environment because they just move the emissions from the car to the power plant"
  • — Same with these people. In Ontario, our electrical grid is very green (about 85% emission free), and even if you live in an area with coal plants it's still better for the environment to drive electric because your total emissions will be reduced and will continue to decrease as those plants are decommissioned.

"I have kids and need a mini-van for car seats, dogs and hockey bags"
  • — Model X solves this one already, but everyone always balks at the price tag. Cybertruck will be a decent answer with a great price, but I know some of my family and friends will still swear they need a Mini-Van option. Amazed no one is making an EV mini-van yet, huge hole in the market that needs to be filled (Chrysler Portal maybe).

While I don't "need" a truck, and I'd prefer an SUV, I already have a Bolt (considered getting a second but I think I'd rather diversify), and while I'd love a Model X, I'm an indie2 app developer in a market where people don't want to pay for apps so I can't justify the cost of an X. The pricing on the Cybertruck is amazing. The dual motor Cybertruck is just over half the cost of the lowest end X. $49,900 vs $84,990 (USD for consistency).

Elon tweeted that a pressurized version of Cybertruck will be used on Mars. I’m not sure if people think he was serious or joking, but I can see it as a real possibility. How cool would it be to drive a Martian truck?

Now, will I actually buy a Cybertruck in late 2021 when they go into production? Which likely means it won't really be available until 2022 (in Canada). That's 2-3 years from now. I actually need a second EV soon since my wife's ICE car is nearing death. We're stretching it out as long as possible, waiting for either the Model Y or the Rivian R1S to ship (to Canada of course). Who knows what my situation will be in 2-3 years when the Cybertruck ships. Maybe one of my apps will take off and I can buy an X. Maybe I'll be working at Apple, Tesla or Rivian (call me 😁). Or maybe I'll be completely happy with whatever I buy in 2020. If stars align and it makes sense, then yes, I'll buy a Cybertruck once it's available. If it was shipping today, it would be on my driveway already. It's not, so the reservation will have to do. The fully refundable deposit is only $150. Compared to the $1000 deposit for the Model 3, this one is chump change. Worst case, it's an interest free loan I'm giving Telsa for a couple of years. Best case, it gets me to the front of the line when they do ship.

Update: just had an interesting interaction that might expose Elon’s hidden genius with Cybertruck. I had a conversation with a gas driver today, someone who has always resisted electrics. I showed them the Cybertruck which they hated the look of, but then they said: "did you see the electric Mustang just announced, I’d rather get that...". Suddenly the question became which electric vehicle to get, not electric vs gas. That’s a serious win!

Footnotes:

1: At some point, we're going to have to pay to remove CO2 from the air. This is a cost that will have to be paid by our governments, and everyone just ignores it and treats it as not their problem. This is especially funny (annoying) because I ran a disguised poll on Twitter asking if someone would pay extra if they caused the damage or just let the everyone else pay, and 92% said they'd pay the extra. But they never do in real life.↩︎

2: I'm also working full time for a big company again, but I'm still indie at heart. ↩︎

My WWDC 2019 Wish List

WWDC 2019


Everyone seems to have their own list of things they want to see at WWDC, so I figured I should throw mine down on virtual paper too. I'll keep it short for you and mostly just include things that aren't on everyone else's lists. [Note: You may notice this is almost exactly like my list from last year, maybe I'll get more of my wishes granted this time around.]

App Stores:

  • ✅ (Partial, tvOS is the exception, of course.) they all get the 2017 update, adding curation etc.
  • (macOS only) allows more powerful (read non-sandboxed) apps back in the store.
  • commission rate change: 5% for apps sold via a deep link, 15% for apps sold via search/browse, 30% for apps sold via curation stories/app lists/features.
  • ability for devs to merge SKUs, i.e., combine X and X Lite into one app. Any user that had downloaded either now gets the merged version and the receipt lets the dev know which one(s) the user originally downloaded. (Odds increase for this one this year, since it could help with Marzipan apps as well.)
  • ability for users to browse all stores on any device, make a purchase, and have the app installed on a different device. I should be able to browse the tvOS AppStore on my Mac, buy a tvOS app and have it install on the family room Apple TV.
  • new badges on every app that indicate features/warnings, such as: age rating, whether or not the app is sandboxed, has passed an accessibility audit, if there's a complimentary macOS/iOS/watchOS/tvOS app, is on your wish list (which they need to bring back), etc. (Hat tip for the accessibility audit idea from Marco Arment on Under the Radar) (Bonus points if there are icons that indicate third party analytics and/or similar frameworks embedded).
AppStore screenshot showing 1Password with new App Badges

iOS:

  • ability to set default apps for email, web, calendar etc.
  • add app shortcuts to Control Center. (Not quite as desired anymore since I work on Launch Center Pro, you should use that instead! 🤪)
  • better control of audio, routing and setting different volumes (ring vs media etc). Rumours suggest something is coming here, hopefully not just a UI change with the same limited functionality.
  • landscape support for Face ID. (Works for iPad Pro now, should work on iPhone too).
  • ✅ multiple faces for Face ID.
  • bring the iPad keyboard to iPhone (the swipe down on a key for the alternate version feature).
  • more granular selection of contacts to allow calls from when in Do Not Disturb mode.
  • multi-user support (for iPhone and iPad).
  • ability to block calls for anyone not in your contact list.
  • fix auto-capitalization. There are a few issues with the way iOS auto-capitalizes letters when typing. First: when iOS determines you need a capital next, you can’t change its mind, for example, let’s say you type “Hi. The dog...”, then realize the period was meant to be a comma, so you use your finger to move the cursor there and change it, then move the cursor to between the T and h, backspace to correct the capital T to a lowercase t. But no, iOS makes it a capital T again, based on the original decision, not based on the current text. Second, and even worse, if using a hardware keyboard, when it decides you need a capital next, there is no way to type a lowercase letter. Tapping the hardware Shift doesn’t undo the pre-pressed software shift. Typing Shift-Letter gives you uppercase, caps lock gives you uppercase. You have to type the letter you want twice, and then delete the first one. (It’s possible this is a bug with the Logitech keyboard I have. Update: Angelo Cammalleri reports this happens with Apple’s keyboard as well).
  • remove the stranglehold on apps, either allow distribution outside of the App Store, or at least stop rejecting apps that Apple doesn’t like. I prefer the walled garden over the Wild West of Android, but perhaps make the walls lower for legit businesses/apps, and higher for the scam apps. If Apple can’t tell the difference, let us crowd-source problem apps.

macOS:

  • the ability to lock the dock to one screen. Having it randomly fly around all my other screens has driven me nuts for years, especially when I go to click an icon on the dock and then the dock runs to a different screen so I can't click it.
  • ✅ HomeKit support
  • fix spaces: when an app has a window in a space and the app crashes, put the window back in the space when it reopens. Safari is the worst for this, I’ll have 20 windows across multiple spaces, it’ll crash, and every window moves to the current space.
  • when booting up, remember which display is where in the arrangement. This has gotten better, but occasionally it still randomly flips displays around on boot up.

tvOS:

  • a built-in web browser.
  • enable UIWebView/WKWebView in tvOS apps.
  • multi-user support.
  • for the love of all that is holy, give tvOS some reason to continue to exist.

watchOS:

  • complications that can update more frequently (1 minute intervals). Even if this requires user permission to update that often.
  • custom watch faces.
  • always on display.

Xcode:

  • plug-in system, at least restoring functionality that was lost in Xcode 8. I'd even be happy with just a way to restore colour to the console logs.

HomeKit:

  • when using automation to turn on a light, be able to turn it off after x number of hours without a second automation. Right now this feature exists, but is limited to 60 minutes. I have several lights that I turn on at sunset, and off at sunrise. They all require 2 automation tasks. Being able to say turn off in 8 hours, would simplify things.
  • display more than just ‘light’ when listing devices in the automation section.

Mac Mini:

  • ✅ (Partial) updated Mac Mini's. Maybe even a Mac Mini Pro with Coffee Lake CPUs, Dual 10 GigE ports, USB-A and C/Thunderbolt 3 ports. Up to 128 GB RAM, 4 TB SSDs. Able to drive 3 5K displays. (We got an update, nearly the exact specs I requested!)

MacBook Pro:

  • ✅ (Partial) updated, with a fixed keyboard design. Coffee Lake CPUs, Up to 64 GB RAM, 4 TB SSDs.
  • option to include the Touch Bar and the standard function keys. I feel most of the hate with the Touch Bar was not with the bar itself, but the removal of the function keys (especially the escape key). I’d buy a MBP that included both.

iPad Pro:

  • ✅ Face ID.

I really wish I could be in San Jose for WWDC this year. It's been a while since I’ve been out with my fellow developers, so you’ll have to have a beer for me. Stay safe, have fun, and hopefully I'll see you next year!


If you've found this article interesting, please subscribe to the RSS feed and follow me on Twitter, Micro.blog and/or Mastodon

It would be awesome if you'd download our newest app All the Rings. It's free and we really think you'll like it!

If you see any errors, want to suggest an improvement, or have any other comments, please let me know.

My WWDC 2018 Wish List

WWDC 2018


Everyone seems to have their own list of things they want to see at WWDC, so I figured I should throw mine down on virtual paper too. I'll keep it short for you and mostly just include things that aren't on everyone else's lists.

AppStores:

  • they all get the 2017 update, adding curation etc.
  • (macOS only) allows more powerful (read non-sandboxed) apps back in the store.
  • commission rate change: 5% for apps sold via a deep link, 15% for apps sold via search/browse, 30% for apps sold via curation stories/app lists/features.
  • ability for devs to merge SKUs, i.e., combine X and X Lite into one app. Any user that had downloaded either now gets the merged version and the receipt lets the dev know which one(s) the user originally downloaded.
  • ability for users to browse all stores on any device, make a purchase, and have the app installed on a different device. I should be able to browse the tvOS AppStore on my Mac, buy a tvOS app and have it install on the family room Apple TV.
  • new badges on every app that indicate features/warnings, such as: age rating, whether or not the app is sandboxed, has passed an accessibility audit, if there's a complimentary macOS/iOS/watchOS/tvOS app, is on your wish list (which they need to bring back), etc. (Hat tip for the accessibility audit idea from Marco Arment on Under the Radar)
AppStore screenshot showing 1Password with new App Badges

iOS:

  • ability to set default apps for email, web, calendar etc.
  • add app shortcuts to Control Center.
  • better control of audio, routing and setting different volumes (ring vs media etc).
  • landscape support for Face ID.
  • multiple faces for Face ID.
  • bring the iPad keyboard to iPhone (the swipe down on a key for the alternate version feature).
  • more granular selection of contacts to allow calls from when in Do Not Disturb mode.
  • multi-user support (for iPhone and iPad).

macOS:

  • the ability to lock the dock to one screen. Having it randomly fly around all my other screens has driven me nuts for years, especially when I go to click an icon on the dock and then the dock runs to a different screen so I can't click it.
  • HomeKit support

tvOS:

  • a built-in web browser.
  • enable UIWebView/WKWebView in tvOS apps.
  • multi-user support.

watchOS:

  • complications that can update more frequently (1 minute intervals). Even if this requires user permission to update that often.
  • custom watch faces.
  • always on display.

Xcode:

  • plug-in system, at least restoring functionality that was lost in Xcode 8. I'd even be happy with just a way to restore colour to the console logs.

HomeKit:

  • when using automation to turn on a light, be able to turn it off after x number of hours without a second automation. Right now this feature exists, but is limited to 60 minutes. I have several lights that I turn on at sunset, and off at sunrise. They all require 2 automation tasks. Being able to say turn off in 8 hours, would simplify things.

Mac Mini:

  • updated Mac Mini's. Maybe even a Mac Mini Pro with Coffee Lake CPUs, Dual 10 GigE ports, USB-A and C/Thunderbolt 3 ports. Up to 128 GB RAM, 4 TB SSDs. Able to drive 3 5K displays.

MacBook Pro:

  • updated, with a fixed keyboard design. Coffee Lake CPUs, Up to 64 GB RAM, 4 TB SSDs.
  • option to include the Touch Bar and the standard function keys. I feel most of the hate with the Touch Bar was not with the bar itself, but the removal of the function keys (especially the escape key). I’d buy a MBP that included both.

iPad Pro:

  • Face ID.

I really wish I could be in San Jose for WWDC this year. It's been a while since I’ve been out with my fellow developers, so you’ll have to have a beer for me. Stay safe, have fun, and hopefully I'll see you next year!


If you've found this article interesting, please subscribe to the RSS feed and follow me on Twitter and/or Micro.blog

It would be awesome if you'd download our newest app All the Rings. It's free and we really think you'll like it!

If you see any errors, want to suggest an improvement, or have any other comments, please let me know.

Mix and Match Swift 3 & Swift 4 Libraries with CocoaPods

With Xcode 9, it’s possible to mix and match Swift 3 and 4 libraries together. In the build settings for the project you set the version of Swift to use by configuring the SWIFT_VERSION setting. You’re able to override the project setting at the target level, thereby building some targets with Swift 3, and others with Swift 4.

If you’re using CocoaPods as your dependancy manager, there’s an issue when mixing and matching.

As you know, when using CocoaPods, you end up with an Xcode workspace that contains your main project, and a Pods project. Whenever you run pod install or pod update, CocoaPods will set the SWIFT_VERSION for all targets to be whatever your main project is set to, or it will fallback to Swift 3 if the main project doesn’t have the SWIFT_VERSION specified.

This means Xcode will try and compile all targets with the same version of Swift, regardless of what version of Swift is actually needed. There’s no built-in way for you to specify the version of Swift to use for each pod you’re including. There is a way for the pod maintainer to specify the version needed in the podspec (they need to set pod_target_xcconfig = { 'SWIFT_VERSION' => '4.0' } see XCGLogger.podspec for an example), but I’ve found it’s rare at this time for it to be set (hopefully this post will help change that).

Even if the pod sets the version of Swift to use, we run into a problem when Xcode resolves the setting. Xcode will prefer the target’s direct setting over the podspec’s suggestion, and since pod update always sets a direct setting on the target, the pod spec’s suggestion is never used (not surprising it’s rarely set).

The solution is to add a post_install script to the end of your podfile:

Let's examine this script.

It’s a post_install script so CocoaPods will execute the script after is has updated all of the included pods and updated the project file.

The script starts by looping through the build configurations of the Pod project and sets the default Swift version to 4.0 (lines 2-5).

Then it loops through all of the project’s targets (lines 7-19). It checks the target name against a known list of targets (line 8) and sets each of the configurations for matching targets to Swift 3 (lines 10-12). If the target isn’t in the known list, the script unsets the Swift version (lines 15-17), which will allow the pod to set the version itself using the pod_target_xcconfig setting we noted above. If the pod doesn’t set the version, Xcode will use the default Swift version we set at the start.

You will need to tweak the script for your project, specifically to set your default Swift version, and then to add the targets that require a different version on line 8.


If you’ve found this article helpful, please subscribe to the RSS feed and follow me on Twitter

It would be awesome if you’d download our new app All the Rings. It’s free and we really think you’ll like it!

If you see any errors, want to suggest an improvement, or have any other comments, please let me know.