"As I mentioned during the meeting with the (EU competition) commissioner, I thought that these decisions had to be applied equally and uniformly," Skvernelis told reporters on Thursday.
"It's a pity that such a decision was made, but still it's, let's say, a step forward, because this company (Gazprom) will have to operate in the market transparently and according to the generally accepted rules," the prime minister said.
"We'll consider (appealing against the decision)," he added.
The EC last year imposed a fine of almost 28 million euros on Lietuvos Geležinkeliai (Lithuanian Railways) for dismantling a railway track to Renge, in Latvia. The state railway company has appealed the decision to the EU's General Court, but has paid the fine.
Russia's gas giant Gazprom may have used its dominant market position to charge higher gas prices in five Central and Eastern European countries, including Lithuania, the EC said on Thursday after completing its anti-trust probe.
The EU's executive body imposed binding obligations on Gazprom. It will face a fine of up to 10 percent of its annual global turnover, which amounted to around 91 billion euros in 2017, if it breaks its commitments.
At Lithuania's initiative, the EC launched the probe into Gazprom's possible infringements in eight member states in 2012. The EC said in April 2015 that the Russian gas giant might have set unfair gas prices and abused its dominant position.