TransWikia.com

Webclient isn't dowloading everything from URL c#

Stack Overflow Asked by Shynex on January 14, 2021

I am trying to get every item name from the steam community market (I market where you buy and sell virtual items, which you can use in games).
Steam has an API link for this, it says "total_count":15227, which represents the number of Items.
But when I try to download it, I get only 100 items. Why is that so?
This is how I download it:

public static Task LoadAllItemsAsync()
{
     using (WebClient w = new WebClient())
     {
          var responseData = w.DownloadString("https://steamcommunity.com/market/search/render/?search_descriptions=0&sort_column=default&sort_dir=desc&appid=730&norender=1&count=500");
           //StatTrak™ FAMAS | Sergeant (Battle-Scarred)
           dynamic parsedJson = JsonConvert.DeserializeObject(responseData);
           string jsonData = JsonConvert.SerializeObject(parsedJson, Formatting.Indented);
                System.IO.File.WriteAllText(System.IO.Path.GetFullPath(@"....SteamDataSteamItems.json"), jsonData);

           return Task.CompletedTask;
      }
 }

One Answer

If you look at the response you can see the "pageSize" parameter that has the value 100.

Apparently 100 is the maximum page size the steam api returns.

{
   "success":true,
   "start":0,
   "pagesize":100,
   "total_count":15227,
   ......
}

So if you want to fetch all items you will need to do a loop. You can use the START and TOTAL_COUNT parameters to calculate the offset etc.

The below URL would skip the first 100 items and return the next 100.

https://steamcommunity.com/market/search/render/?search_descriptions=0&sort_column=default&sort_dir=desc&appid=730&norender=1&count=100&start=100

Correct answer by JOSEFtw on January 14, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP