I have an issue where i am trying to query a dataset of over 3000 items and return the results to a drop down menu. The issue i'm having is that the query will only lookup the 1st 1000 rows of the dataset. Below is my code, how do i look up the full 3000 rows and return the results to a single drop down menu ?
import wixData from 'wix-data'; $w.onReady(function () { uniqueDropDown(); }); function uniqueDropDown() { wixData.query("Collection_Database") .limit(1000) .ascending("Column1") .find() .then(results => { const uniqueTitles = getUniqueTitles(results.items); $w("#uniqueDropDown").options = buildOptions(uniqueTitles); }); function getUniqueTitles(items) { const titlesOnly = items.map(item => item.Column1); return [...new Set(titlesOnly)]; } function buildOptions(uniqueList) { return uniqueList.map(curr => { return { label: curr, value: curr }; }); }
If you wanted to fetch 3000 elements, you could:
async function fetchData() { const firstPage = await wixData.query('collection') .limit(1000) .find(); const secondPage = await wixData.query('collection') .limit(1000) .skip(1000) .find(); const thirdPage = await wixData.query('collection') .limit(1000) .skip(2000) .find(); const allItems = firstPage.items.concat(secondPage.items).concat(thirdPage.items); return allItems; }
By the way, I am using async / await feature, to make it more readable compared to nesting .then(...) statements.
There is also an alternative way to do this using next().
async function fetchData() { let result = await wixData.query('collection') .limit(1000) .find(); // Result is WixDataQueryResult and has next() method let allItems = result.items; while (result.hasNext()) { // Repeat while there is a next page result = await result.next(); // We fetch next page the easy way, without using skip and limit allItems = allItems.concat(result.items); // We add results of the next page to allItems } return allItems; // In the end we return the list of items from all pages }
This will fetch all of the items in the collection, no matter the count. However, for a sufficiently large collection, this might take a while, so be sure not to use it with very large collections as the performance of the site will suffer.
https://www.wix.com/code/reference/wix-data.WixDataQueryResult.html
As a developer and an analyst dealing with large data sets, having a limit is really annoying. But, that being said, having a limit is also beneficial when trying to bring all that data onto the screen. I find anymore than 100 visible rows tends to really slow the performance.
My recommendation would be to review your data structure and build efficient indexing using relational data.
thanx your a legend mate!
It took me a few while to get it working but finally got it working today. Thank you very much for this.
Hi All!
The limit of a query is by default = 50 with a maximum of 1000 returned values.
It's currently not possible to return more than 1000 items at once.
You can work around it by adding the .skip() method to your query and set the number of items you want to skip before the query starts to return items from the collection.
Hope it helps!
Doron.
That is a Lot of items to search and would cause a freeze (can take 5 + seconds on my site with 50 items) ,
one alternative could be multiple pages of items in the dropdown?
They are not scrolling down through 3000 items in the dropdown list, as you can see from my above code I am only showing unique values in the dropdown. There are 3000 items that have to be queried in the dataset but only unique items are returned to the dropdown.
There may be a limit in Wix WRT the # of items in a dropdown list ...
Even if there isn't, 3000 is a lot of items for a user to scroll through to select from; you could first filter down the list ...
No that does not enable the query to search 3000 items. Similarly if i change "1000" to "3000" then it does not return any results it just freezes.
Trying removing the following from your code:
.limit(1000)
and see if that does what you want.