Android GridView update background Previous click

I am changing the color of the cells in a GridView depending on some comparisons. I have a method that checks if the number is closest to a randomly generated number, then I paint yellow if the selection is closer than the ones that have already been clicked. But I want the previous closest number to be painted red, so as not to have several cells in yellow, only two. Code that compares and paints the cells:

gridViewNumbers.setOnItemClickListener(new AdapterView.OnItemClickListener() {
    @Override
    public void onItemClick(AdapterView<?> parent, View view, int position, long id) {
        Toast.makeText(MainActivity.this,"Random number is: " +randomNumber +" number is: " +listNumbers[position], Toast.LENGTH_SHORT).show();

        int selectedNumber = listNumbers[position];

        numbersCloser(selectedNumber);

        if (selectedNumber == randomNumber) {
            view.setBackgroundColor(Color.GREEN);
        }

        if (selectedNumber == numberBigger || selectedNumber == numberSmaller) {
            view.setBackgroundColor(Color.YELLOW);
        }

        if (selectedNumber != randomNumber && selectedNumber != numberBigger && selectedNumber != numberSmaller) {
            view.setBackgroundColor(Color.RED);
        }

        numberLastPlay = selectedNumber;
    }

});

Code that updates the nearest larger and smaller number:

private int numbersCloser(int selected){
    if (selected < numberBigger && selected > randomNumber){
        return numberBigger = selected;
    }

    if (selected > numberSmaller && selected < randomNumber){
        return numberSmaller = selected;
    }
    else{
        return 0;
    }
}

I think I'll have to create variables to keep the old numbers nearby or change the color to red in the cells before updating the next number variables. Any suggestions on how I can do?