Looks like it now has Docling Content Extraction Support for RAG. Has anyone used Docling much?

It really depends on how you quantize the model and the K/V cache as well. This is a useful calculator. https://smcleod.net/vram-estimator/ I can comfortably fit most 32b models quantized to 4-bit (usually KVM or IQ4XS) on my 3090’s 24 GB of VRAM with a reasonable context size. If you’re going to be needing a much larger context window to input large documents etc then you’d need to go smaller with the model size (14b, 27b etc) or get a multi GPU set up or something with unified memory and a lot of ram (like the Mac Minis others are mentioning).

It would be cool if they would provide some useful statistics about the aggregated data as well. Maybe something like showing the percentile for pay to the ED/CEO or for the total compensation compared to other organizations in the sector.

I didn’t scour the site so maybe this does exist.