Mojeek Search Engine

for the moment it’s just not in the crawl; I will add +1 into the “add Japanese” column off the back of this interaction

haha, now do openai

which languages are you searching in? It’s likely that they are as-of-yet unsupported. There are a lot of languages we crawl and index in which aren’t en/de/fr but non-neo-latin languages are not currently available.

That would be helpful, yes; there is a “submit feedback” button on results pages which allows you to send it directly to us if that works.

Mojeek never finds anything.

Got any examples we can look into?

do you have any examples? they’re always useful for us when it comes to targetting improvements etc.

“hello i am u/NotBillGates and I agree with this message”

probably a sample size issue, we crawl and index everything we are able to; have seen many of this kind of site in the past, and finding them is something that other people have said they enjoy about mojeek

our preference is always to find out why the block is happening and try to convince people it should be otherwise; widespread abuse of robots.txt does no-one any good, having been crawling and indexing for so long it’s a standard that we understand and are quite fond of

we can see some of the perils and pitfalls of it too, but web builders need to be given some tools and assurances that those tools will work for them

IP already hits a wall, also better to not get a reputation as a bad bot, it’s taken a while to get known for being friendly and respecting rules, to us you should follow robots

We make money from our API, what they’re referencing is a beta ads programme which was running

Agree to disagree here, but I’ll refer to Cory Doctorow for a contextual vs behavioral/tracking ads comparison, one which is very good: https://pluralistic.net/2020/08/05/behavioral-v-contextual/#contextual-ads (applied to the media, but the general thread is relevant)

which language are we talking?

Reddit doesn’t allow us to crawl: https://www.reddit.com/robots.txt

qwant is bing, mainly

we make money mainly from our api, our investors are patient private capital and we don’t take vc, appreciate your point but these are fundamentally different situations, our ads (when they run) will also be contextual so more of a ddg situation than a “makes users into products to be sold to advertisers”

fair enough if it’s not for you though

ah no bother at all, not everyone is gonna be across every single kind of company and they are functionally very similar!

good write up, we’re Mojeek Limited, not Mojeek LLC; LLC as a formation in the UK is called a “Private Limited Company” (PLC) which is contracted to .ltd or limited.