Before cheering supporters, Mr Tsipras again pledged to renegotiate Greece’s massive international bailout.
With nearly all of the votes counted in Sunday’s poll, Syriza looks set to have 149 seats, just two short of an absolute majority.
Syriza’s victory has raised fears about Greece’s future in the euro.
The currency fell to $1.11 against the US dollar following the result – the lowest level in more than 11 years.
Greece has endured tough budget cuts in return for its 2010 bailout, worth €240bn (£179bn; $268bn) and negotiated with the so-called troika – the European Union, International Monetary Fund (IMF) and European Central Bank (ECB).
The economy has shrunk drastically since the 2008 global financial crisis, and increasing unemployment has thrown many Greeks into poverty.
Syriza’s election result will send shockwaves through Europe, the BBC’s Gavin Hewitt in Athens says.
A majority of voters in Greece have essentially rejected a core policy for dealing with the eurozone crisis as devised by Brussels and Germany, our correspondent adds.
The election result is expected to be one of the main issues at Monday’s meeting of 19 eurozone finance ministers.
In Germany, Bundesbank President Jens Weidmann said he hoped “the new Greek government will not make promises it cannot keep and the country cannot afford”.
Belgian Finance Minister Johan Van Overtveld was quoted by VRT network as saying that Greece “must respect the rules of monetary union”, although he added that there was room for some flexibility.
UK’s Chancellor of the Exchequer (Finance Minister) George Osborne told the BBC: “Ultimately if you take at face value all the things that the new Greek government has promised – including big increases in public expenditure – you know, I think that is going to be very difficult to deliver, and incompatible with what the eurozone currently demands of its members.”
In Italy, EU Affairs Minister Sandro Gozi said: “After this vote we will have new opportunities to pursue change in Europe to create growth and investment and fight against unemployment.”
Results from Greece’s election commission showed a clear Syriza lead.
With most votes counted, Syriza polled 36%, while the outgoing New Democracy won 28%.
Another five parties – including the far-right Golden Dawn and centrist The River – are expected to be represented in the 300-member parliament.
Syriza is thought likely to begin talks with smaller parties on forming a government.
Outgoing Prime Minister Antonis Samaras has admitted defeat.
Addressing his cheering supporters in Athens late on Sunday, Mr Tsipras said “the Greeks wrote history”.
“Greece is leaving behind catastrophic austerity, it is leaving behind the fear and the autocracy, it is leaving behind five years of humiliation and pain”.
“Your mandate is undoubtedly cancelling the bailouts of austerity and destruction.
“The troika for Greece is the thing of the past,” he added.
Mr Tsipras promised to write off half of Greece’s debt, but was ready to negotiate “a viable solution” and wants the country to stay in the eurozone.
For his part, Mr Samaras said: “The Greek people have spoken and I respect their decision,” pointing out that he had inherited a “hot potato” on coming into office and that he and his party had done much to restore his country’s finances.