mlx-community/EdgeTAM-fp16

EdgeTAM is on-device SAM 2 for promptable segmentation and video tracking. This repo holds it converted to Apple MLX (fp16) for the mlx-edgetam-swift Swift package (EdgeTAMPackage, an MLXEngine promptSegment + trackObject ModelPackage). There are 874 tensors (image + video), checked against the official edgetam.pt (fp16 rounding only).

Parity (v0.5.0). Measured on the CPU fp32 stream against upstream PyTorch, through the public predictors:

• image: image_embed relative error ≤ 2.4e-4, and every click and box mask matches at IoU 1.0000 on square and non-square images;
• video: 5-frame point and box tracks stay at IoU ≥ 0.999.

Expect weak single clicks on flat-shaded or synthetic images. Upstream EdgeTAM behaves the same way; a box prompt is its reliable input. For a better click-selection tier see SAM2.1-hiera-small-fp16.

Use

// .package(url: "https://github.com/xocialize/mlx-edgetam-swift", from: "0.6.0")
import EdgeTAM
let p = try EdgeTAMPredictor.fromPretrained(weightsPath, dtype: .float16)
p.setImage(sourceCGImage)                       // encoder once per image
let r = p.predict(point: (500, 375))            // r.mask, r.soft (anti-aliased), r.score
let vp = try EdgeTAMVideoPredictor.fromPretrained(weightsPath, dtype: .float16)

Weights: Apache-2.0 (facebookresearch/EdgeTAM). Port code: MIT.