Category
🤖
TechTranscript
00:00So a couple days ago, Apple announced this super cool feature where they used an iPhone and they connected it up to a MacBook
00:05and they were able to create this top-down image using the ultrawide camera on this phone.
00:11So in the demo, they have a phone mounted on top of an Apple Pro Display XDR
00:15and according to Apple, the image would look something like this.
00:18A top-down shot of the desk space in front of you, captured from the ultrawide camera of that phone.
00:25And that image looks very impressive, and I think a lot of people that saw that were like,
00:29that is super cool, but it's like, is that actually how good it would look?
00:33Like, that looks almost too good to be true.
00:36And then, they followed up with this image, which looks even more impossible,
00:40where they have the phone mounted onto a MacBook, which is a shorter device, so it's closer to the table,
00:46and in that picture, it's tilted back a little bit, so it's an even steeper angle for the camera to pull off this trick.
00:51And now, it has to process this heavily distorted image in real time to create this video feed of this top-down looking shot.
00:59And I thought, as impressive as this stuff is, like, there's no way, right?
01:02There's no way it can possibly look as good as the media images.
01:07Well, there's a way to test this, because Apple allows their developers to test their upcoming software features.
01:13So, I downloaded the new Xcode beta, I downloaded iOS 16 and the macOS 13,
01:18and I compiled their Continuity Camera app to test this myself.
01:21And, oh my god, there's a lot to talk about.
01:25So, first, I don't have that fancy mount that you see in the media photos, because it doesn't exist yet.
01:31So, instead, I decided to use duct tape.
01:35And who's even gonna buy that mount, when it can look like this with a little bit of work, right?
01:41Okay, so desk view in Continuity Camera looks really good, but there's a few things I noticed.
01:46First, your hands and fingers can look really weird when they come into the shot.
01:50Like, sometimes it looks like a bunch of sausages.
01:53But it also works best when you just have a flat object or surface that you're trying to show.
01:56So, if you're writing on a piece of paper, that can look really good.
01:59Or if you're doing some kind of card tricks, that can work really well.
02:01Anything that's flat will look like a legitimate overhead camera shot.
02:05But if there's any height to the object, that's when things can look a little whack.
02:10Like this book here, it's not a particularly tall book, but when I bring it into the scene, like, you can't even see the top of the book.
02:16You can only see the spine in the shot.
02:18Like, the top of the book, it's invisible.
02:20Like, you gotta bring it all the way in.
02:22You know what?
02:23You can never see the top of the book.
02:25Well, there.
02:26Like, when it's on the computer, like, it's just a, it's a really steep angle.
02:30It's got a good algorithm.
02:32Like, my fingers don't look too weird, right?
02:36But it's gotta be a flat object.
02:38The flatter, the better.
02:40Another thing I noticed is that this top-down desk view works best when the phone is mounted in a portrait position, like a vertical position.
02:47So I taped it to the back of this device in, like, the landscape horizontal mode.
02:53And it works, but you get a way bigger field of view when it's portrait.
02:59And the reason being, I think, is that that's just the way the sensors work, right?
03:03The sensor inside a camera is a rectangle when it's in that upright position.
03:07Because you do get a pretty serious sensor crop.
03:09When it's in this landscape mode, it's a nice wide shot.
03:12The last thing I noticed is image quality.
03:14So in this kind of desk view mode, the image isn't great.
03:19It's, like, a cool-looking image in terms of, like, the technology behind it.
03:23But it's, like, low-res and the colors aren't great.
03:26And same thing with center stage.
03:27When you have that enabled and it's got that cool face-tracking thing and you can move around and it'll track your face, very cool.
03:34But the image quality is reduced.
03:36Compared to when you turn off center stage and instead of using the ultra-wide,
03:41because all that stuff uses the ultra-wide lens,
03:43when you turn it off, it switches over to the main sensor.
03:45But you got to tilt it up.
03:46But look at the image quality.
03:48Oh, my God.
03:49Like, look at that.
03:50This is amazing for a webcam.
03:54I mean, it's not really a webcam.
03:55It's a phone that's stuck to a computer that's serving as a webcam.
03:57But, like, look at the image quality here and you compare it to any other traditional webcam.
04:01I don't even know which one I'm looking at.
04:02Okay, it's this one.
04:03It's the top left one.
04:05That's the main sensor.
04:07But, like, look at the image quality on this.
04:09This is phenomenal for a webcam-like experience.
04:14Now, there's also a portrait mode and a studio light mode that's already functional in this beta.
04:18And I'll just enable it real quick.
04:20The portrait mode, it gives you, like, a bokeh, like, in the back of your shot.
04:25Like, it'll just blur it out.
04:26It's still pretty artificial-looking to me, but I don't know if that's your jam.
04:30But there's also the studio light mode.
04:32So this is actually really neat.
04:33It's good for kind of poor lighting conditions, whether it's dark everywhere or really bright in the background.
04:41It'll just illuminate your face in a manner to just make it a better shot.
04:45Like, if I turn it off real quick here.
04:47I mean, this studio is so well lit that it doesn't make that much of a difference visually.
04:52But if you turn off the portrait mode, you turn off that bokeh, and you put on the studio light.
04:58That's pretty cool.
04:59I think it's pretty neat.
05:01But it's most noticeable when you have a really bright background behind you.
05:04Like, let's say you're against a window, or if you're, like, outdoors, and there's, like, bright light behind you.
05:09Normally, your face is underexposed, right?
05:11Because it's so bright up there, and it's having a tough time capturing your face properly, like the camera.
05:16You hit on studio light.
05:18It'll illuminate your face, and it does a good job.
05:20Now, I think as early as this is, like, this is, I don't know, day two of the beta.
05:25Like, they've done a fantastic job with this whole thing.
05:28So there you have it.
05:29Continuity camera with desk view.
05:31It really does work the way that Apple said it would.
05:34You just got to use flat objects.
05:35Because if you don't, and you're using something big, like two thumbs up, it just looks a little weird.