In a Facebook post on Tuesday, Gavrilovas said he had answered recent questions concerning his loyalty to Lithuania.
"However, the endless debates and doubts about my ability to perform my current duties are not helping the ministry to work effectively. So, I have taken the decision to step down as deputy minister of finance," he wrote.
Meanwhile, Finance Minister Rimantas Šadžius reiterated that he has confidence in Gavrilovas. He said that his colleague, as a citizen and as a politician, had made a selfless decision to stand down.
"As I have already said, I had no reason to doubt Valentinas Gavrilovas’ loyalty to Lithuania and his professional competence, but the public’s expectation of politicians is quite rightly higher than that alone," the minister is cited as saying in a press release.
Several years ago, Gavrilov posted a message on Facebook congratulating people on 9 May, the Victory Day. Commenting to public broadcaster LRT, he said he was loyal to Lithuania but would continue celebrating 9 May, just a day earlier, on 8 May, as it is important to his family.
Russia celebrates Victory Day on 9 May, whereas the West marks Victory in Europe Day on 8 May.
When the Soviet Union occupied Lithuania, its forces did not leave after the Second World War, instead carrying out repressions against the civilian population.
Gavrilovas was again in the spotlight on Monday, when LRT Radio reported that the politician had an account on Russian social network VKontakte where he shared songs celebrating Russia, its army and the Soviet Union. The deputy minister said he had already deleted the account.