Filter information in python dictionaries dynamically

1

First of all, I want to say that I am understanding how to filter the information in the Python dictionaries from a JSON and then write them in a JSON file. I take a normal workflow on the basis that we do not always know exactly what we should iterate. To illustrate a simple but real-life example.

The workflow

  • Decode JSON
  • Treat and process the data (convert it into information)
  • Encode JSON and write them in a file

The example

For this I am using the CoinMarketCap apin and this is the data structure:

{
  "data": {
    "1": {
      "id": 1,
      "name": "Bitcoin",
      "symbol": "BTC",
      "website_slug": "bitcoin",
      "rank": 1,
      "circulating_supply": 17168112.0,
      "total_supply": 17168112.0,
      "max_supply": 21000000.0,
      "quotes": {
        "USD": {
          "price": 8159.91,
          "volume_24h": 6805930000.0,
          "market_cap": 140090248790.0,
          "percent_change_1h": -0.65,
          "percent_change_24h": 5.84,
          "percent_change_7d": 18.14
        }
      },
      "last_updated": 1532445803
    },
    "1027": {
      "id": 1027,
      "name": "Ethereum",
      "symbol": "ETH",
      "website_slug": "ethereum",
      "rank": 2,
      "circulating_supply": 100891486.0,
      "total_supply": 100891486.0,
      "max_supply": null,
      "quotes": {
        "USD": {
          "price": 472.418,
          "volume_24h": 2187530000.0,
          "market_cap": 47662953974.0,
          "percent_change_1h": -1.42,
          "percent_change_24h": 2.14,
          "percent_change_7d": -1.83
        }
      },
      "last_updated": 1532445813
    },
    "52": {
      "id": 52,
      "name": "XRP",
      "symbol": "XRP",
      "website_slug": "ripple",
      "rank": 3,
      "circulating_supply": 39315683476.0,
      "total_supply": 99991900487.0,
      "max_supply": 100000000000.0,
      "quotes": {
        "USD": {
          "price": 0.458599,
          "volume_24h": 305718000.0,
          "market_cap": 18030133126.0,
          "percent_change_1h": -1.22,
          "percent_change_24h": 1.69,
          "percent_change_7d": -5.25
        }
      },
      "last_updated": 1532445797
    },
  "metadata": {
    "timestamp": 1532445415,
    "num_cryptocurrencies": 1664,
    "error": null
  }
}

As you can see are nested dictionaries and there comes a point where you can not iterate explicitly, we must do it dynamically, otherwise we should know the identification of the encryption currency.

We begin the exploration of the dictionaries:

#!/usr/bin/python3

import json
import requests

response = requests.get("https://api.coinmarketcap.com/v2/ticker/")
data = json.loads(response.text)


def keep(data):
    for i in data['data'].key():
        print(i)


keep(data)    

As you can see, this is the complex part to iterate.

I managed to get rid of the complex part to get the Top10 of cryptocurrencies:

#!/usr/bin/python3

import json
import requests

response = requests.get("https://api.coinmarketcap.com/v2/ticker/")
data = json.loads(response.text)


def keep(data):
    for i in data['data'].values():
        if i['rank'] <= 10:
            print(json.dumps(i, indent=4))


keep(data)    

But I can not write it in a JSON file, I hope you can help me, thanks in advance.

    
asked by Nelson 24.07.2018 в 18:28
source

1 answer

1

To create your JSON file with the "Top10" you only have to filter as you do but instead of trying to serialize each item separately, add it to a list or dictionary, which you then serialize with jsom.dump (to create a file. json) or with json.dumps to create a string representing one:

top_10 = [i for i in data['data'].values() if i['rank'] <= 10]
with open('top10.json', 'w') as fp:
    json.dump(top_10, fp, indent=4, separators=(',', ': '))

With what you get a file with the following structure:

[
    {
        "id": 1,
        "name": "Bitcoin",
        "symbol": "BTC",
        "website_slug": "bitcoin",
        "rank": 1,
        "circulating_supply": 17168400.0,
        "total_supply": 17168400.0,
        "max_supply": 21000000.0,
        "quotes": {
            "USD": {
                "price": 8231.71,
                "volume_24h": 6899640000.0,
                "market_cap": 141325289964.0,
                "percent_change_1h": 0.68,
                "percent_change_24h": 6.51,
                "percent_change_7d": 17.23
            }
        },
        "last_updated": 1532453664
    },
    {
        "id": 1027,
        "name": "Ethereum",
        "symbol": "ETH",
        "website_slug": "ethereum",
        "rank": 2,
        "circulating_supply": 100893214.0,
        "total_supply": 100893214.0,
        "max_supply": null,
        "quotes": {
            "USD": {
                "price": 475.778,
                "volume_24h": 2195980000.0,
                "market_cap": 48002771511.0,
                "percent_change_1h": 0.55,
                "percent_change_24h": 3.94,
                "percent_change_7d": -2.2
            }
        },
        "last_updated": 1532453675
    },

    ....
]

Now if you want to keep the structure of the original JSON you can do the following:

top_10 = {"data": {k: v for k, v in data['data'].items() if v['rank'] <= 10}}
with open('top10.json', 'w') as fp:
    json.dump(top_10, fp)

With what you get a file like the following:

{
    "data": {
        "1": {
            "id": 1,
            "name": "Bitcoin",
            "symbol": "BTC",
            "website_slug": "bitcoin",
            "rank": 1,
            "circulating_supply": 17168400.0,
            "total_supply": 17168400.0,
            "max_supply": 21000000.0,
            "quotes": {
                "USD": {
                    "price": 8214.46,
                    "volume_24h": 6881340000.0,
                    "market_cap": 141029135064.0,
                    "percent_change_1h": 0.46,
                    "percent_change_24h": 6.3,
                    "percent_change_7d": 17.05
                }
            },
            "last_updated": 1532453368
        },
        "1027": {
            "id": 1027,
            "name": "Ethereum",
            "symbol": "ETH",
            "website_slug": "ethereum",
            "rank": 2,
            "circulating_supply": 100893214.0,
            "total_supply": 100893214.0,
            "max_supply": null,
            "quotes": {
                "USD": {
                    "price": 474.556,
                    "volume_24h": 2187020000.0,
                    "market_cap": 47879480003.0,
                    "percent_change_1h": 0.22,
                    "percent_change_24h": 3.63,
                    "percent_change_7d": -2.42
                }
            },
            "last_updated": 1532453376
        },
        "52": {
            "id": 52,
            "name": "XRP",
            "symbol": "XRP",
            "website_slug": "ripple",
            "rank": 3,
            "circulating_supply": 39315683476.0,
            "total_supply": 99991900487.0,
            "max_supply": 100000000000.0,
            "quotes": {
                "USD": {
                    "price": 0.456089,
                    "volume_24h": 308807000.0,
                    "market_cap": 17931450761.0,
                    "percent_change_1h": -0.14,
                    "percent_change_24h": 1.9,
                    "percent_change_7d": -7.08
                }
            },
            "last_updated": 1532453356
        },
        "1831": {
            "id": 1831,
            "name": "Bitcoin Cash",
            "symbol": "BCH",
            "website_slug": "bitcoin-cash",
            "rank": 4,
            "circulating_supply": 17254788.0,
            "total_supply": 17254788.0,
            "max_supply": 21000000.0,
            "quotes": {
                "USD": {
                    "price": 855.979,
                    "volume_24h": 861947000.0,
                    "market_cap": 14769735749.0,
                    "percent_change_1h": 0.67,
                    "percent_change_24h": 7.56,
                    "percent_change_7d": 4.28
                }
            },
            "last_updated": 1532453376
        },
        "1765": {
            "id": 1765,
            "name": "EOS",
            "symbol": "EOS",
            "website_slug": "eos",
            "rank": 5,
            "circulating_supply": 896149492.0,
            "total_supply": 900000000.0,
            "max_supply": 1000000000.0,
            "quotes": {
                "USD": {
                    "price": 8.48947,
                    "volume_24h": 966135000.0,
                    "market_cap": 7607834229.0,
                    "percent_change_1h": 1.27,
                    "percent_change_24h": 6.67,
                    "percent_change_7d": 1.49
                }
            },
            "last_updated": 1532453377
        },
        "512": {
            "id": 512,
            "name": "Stellar",
            "symbol": "XLM",
            "website_slug": "stellar",
            "rank": 6,
            "circulating_supply": 18767299129.0,
            "total_supply": 104144920420.0,
            "max_supply": null,
            "quotes": {
                "USD": {
                    "price": 0.300038,
                    "volume_24h": 106479000.0,
                    "market_cap": 5630902896.0,
                    "percent_change_1h": -0.03,
                    "percent_change_24h": 5.2,
                    "percent_change_7d": 24.63
                }
            },
            "last_updated": 1532453365
        },
        "2": {
            "id": 2,
            "name": "Litecoin",
            "symbol": "LTC",
            "website_slug": "litecoin",
            "rank": 7,
            "circulating_supply": 57548957.0,
            "total_supply": 57548957.0,
            "max_supply": 84000000.0,
            "quotes": {
                "USD": {
                    "price": 87.8045,
                    "volume_24h": 411849000.0,
                    "market_cap": 5053057399.0,
                    "percent_change_1h": 0.54,
                    "percent_change_24h": 4.48,
                    "percent_change_7d": 1.57
                }
            },
            "last_updated": 1532453350
        },
        "2010": {
            "id": 2010,
            "name": "Cardano",
            "symbol": "ADA",
            "website_slug": "cardano",
            "rank": 8,
            "circulating_supply": 25927070538.0,
            "total_supply": 31112483745.0,
            "max_supply": 45000000000.0,
            "quotes": {
                "USD": {
                    "price": 0.17286,
                    "volume_24h": 191886000.0,
                    "market_cap": 4481753413.0,
                    "percent_change_1h": -0.03,
                    "percent_change_24h": 2.89,
                    "percent_change_7d": 7.56
                }
            },
            "last_updated": 1532453378
        },
        "1720": {
            "id": 1720,
            "name": "IOTA",
            "symbol": "MIOTA",
            "website_slug": "iota",
            "rank": 9,
            "circulating_supply": 2779530283.0,
            "total_supply": 2779530283.0,
            "max_supply": 2779530283.0,
            "quotes": {
                "USD": {
                    "price": 0.979802,
                    "volume_24h": 46447800.0,
                    "market_cap": 2723389330.0,
                    "percent_change_1h": 0.61,
                    "percent_change_24h": 1.93,
                    "percent_change_7d": -11.04
                }
            },
            "last_updated": 1532453372
        },
        "825": {
            "id": 825,
            "name": "Tether",
            "symbol": "USDT",
            "website_slug": "tether",
            "rank": 10,
            "circulating_supply": 2507140346.0,
            "total_supply": 3080109502.0,
            "max_supply": null,
            "quotes": {
                "USD": {
                    "price": 0.996441,
                    "volume_24h": 4194340000.0,
                    "market_cap": 2498217433.0,
                    "percent_change_1h": -0.15,
                    "percent_change_24h": -0.19,
                    "percent_change_7d": -0.48
                }
            },
            "last_updated": 1532453373
        }
    }
}
    
answered by 24.07.2018 / 19:39
source