No system is reliable without validation so we started adding validations in our vector database migration tool to make it more secure.

There was something we did not initially expect to be a major issue.

During a migration, all the records could be moved successfully to the target database, yet the search results were not always the same.

That sounds surprising at first, but it is a fairly common problem with vector database migrations. It is also easy to miss because most migration checks focus on whether the records, IDs and metadata were copied successfully.

For example, you could move data from Qdrant, Zilliz, or between other vector databases Weaviate, Chroma, LanceDB, MongoDB, Databricks, see that everything has migrated successfully and assume the migration is complete. But the search behaviour can still change without being obvious.

So we started looking at the migration differently:

A database migration can copy every record successfully, but still change what search returns.

The reason is partly in the similarity configuration. Different vector databases can use different similarity settings, and those settings affect how vectors are compared, selected and ranked.

We added backend checks to read these settings from both the source and destination databases, compare them, and include the findings in the validation report shown in MING’s UI.

There was another detail we cared about here. If a particular setting could not be read, we did not want the system to silently skip it. The report shows the limitation and explains why that setting could not be checked.

This means a search configuration issue can now be identified even when the record counts and migration checks look completely fine.

There was another part we worked on after that.

Once we could show that search results were different, the next question was obvious: why?

So we added a diagnostic step to the migration process.

For different database pairs, such as Pinecone to Qdrant or Weaviate to Qdrant, we take candidate results from the destination, retrieve the original vectors from the source, and evaluate them using the source similarity metric.

The frontend then presents these diagnostic findings alongside the main validation report, so the difference can be investigated in the same place.
And when the evidence is not strong enough to explain the difference, we do not try to guess. The result is marked as inconclusive.

That gives the user something more useful than a low overlap score. It provides evidence about where the difference may be coming from, while being clear about what the system can and cannot determine.

We are still working on a few automations in this part and will share with you how it goes!